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transformation  rules  from  a PROOF  code  to  the  1PR  and  from  the  IPR  to  a target 
code  and  show  that  the  transformation  rules  preserve  the  correctness  of  the  origi- 
nal PROOF  programs.  For  efficient  exploitation  of  parallelism,  we  present  the  grain 
size  determination  algorithms  on  various  patterns  of  parallelism  and  compare  them 
with  the  existing  approaches.  Software  development  based  on  PROOF  has  the  fol- 
lowing distinct  advantages:  First,  parallel  aspects  of  the  software  can  be  treated  as 
a prime  issue,  using  object-oriented  concepts.  Second,  our  approach  can  reduce  the 
communication  and  synchronization  errors  by  automatically  embedding  proper  codes 
during  the  transformation  sUge.  Third,  the  use  of  the  IPR  improves  portability  of 
the  software  by  separating  architecture-dependent  issues  from  the  semantics  of  the 


software. 


CHAPTER  1 
INTRODUCTION 

With  the  advances  in  computer  technology,  the  speed  of  computers  has  signifi- 
cantly increased  over  the  last  several  decades.  However,  even  such  vastly  increased 
speed  is  not  sufficient  to  fulfill  our  desire  to  solve  complex  problems,  involving  artifi- 
cial intelligence,  robotics,  high  energy  physics,  molecular  physics  and  space  sciences. 
Parallel  processing  appears  to  be  a promising  solution  to  satisfy  such  problems.  Un- 
fortunately, software  development  methods  and  techniques  for  parallel  processing 
systems  are  far  from  being  mature  enough  to  effectively  utilise  their  potential.  The 
lack  of  such  methods  and  techniques  is  a major  obstacle  to  the  widespread  use  of 
parallel  processors  in  a variety  of  application  areas.  The  goal  of  this  research  is  to 
develop  a software  development  approach  for  parallel  processing  systems  and  study 
the  issues  involved  in  such  development. 

Software  development  is  more  complex  for  parallel  processors  than  for  sequential 
computers  in  at  least  two  aspects:  program  correctness  and  efficiency.  One  of  the 
requirements  for  the  correctness  of  the  parallel  programs  is  the  independence  of  the 
execution  results  on  the  number  and  speed  of  processors  running  the  program.  This 
requirement  can  be  met  only  if  the  parallel  tasks  arc  properly  synchronized.  The 
focus  on  efficiency  in  parallel  program  design  increases  the  complexity  of  software 
development.  The  efficiency  of  parallel  implementation  is  usually  measured  by  di- 
viding the  speed-up  of  the  parallel  program  relative  to  its  sequential  version  with 
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the  number  of  processors  involved  in  that  parallel  program  execution.  Thus,  in  or- 


software  development  for  parallel  processing  systems  need  to  be  developed. 


Parallel  processing  of  a program  is  based  on  the  following  two  assumptions: 

• the  availability  of  the  parallel  computers  that  can  cooperate  to  solve  a given 


• the  existence  of  program  development  techniques  such  as  design  techniques  for 
parallel  programs  and  efficient  implementation  strategies. 

In  fact,  as  cost  effective  parallel  computers  become  more  available  on  the  market, 

to  be  met.  However,  current  software  development  techniques  for  parallel  processing 
systems  are  not  adequate  to  specifically  address  the  issues  involved  in  programming 
parallel  processors,  most  importantly  parallelism. 

The  essence  of  parallel  processing  is  to  manage  parallelism.  Parallelism  refers 
to  the  execution  of  some  portions  of  a program  in  a simultaneous  manner.  The 
management  of  the  parallelism  involves  two  issues: 

• Expressing  parallelism:  for  parallel  processing  of  a program,  the  parallelism  in 
that  program  needs  to  be  represented  using  a programming  language,  and 

• Exploiting  parallelism:  the  parallelism  needs  to  be  efficiently  exploited  on  the 
underlying  parallel  computers. 

Parallelism  can  be  specified  either  implicitly  or  explicitly.  Explicit  parallelism  is 
expressed  using  explicit  parallel  constructs  by  the  programmers.  Thus,  the  detec- 
tion of  the  explicit  parallelism  is  easy.  However,  the  programmers  have  to  ensure 


der  to  achieve  a i 


: in  terms  of  efficiency,  proper  techniques  for 


problem, 


the  correct  use  of  such  constructs  in  terms  of  communication  and  synchronization  of 
parallel  programs.  In  contrast,  implicit  parallelism  implies  that  programmers  are  not 
concerned  about  the  parallelism.  In  this  case,  the  underlying  computation  models 
inherently  support  the  parallelism.  Thus,  programmers  do  not  use  any  explicit  par- 
allel constructs.  On  the  other  hand,  cither  a compiler  or  run-time  system  requires  to 
detect  parallelism  in  the  programs.  Once  the  parallelism  is  explicitly  expressed  and 
detected,  it  needs  to  be  exploited  on  parallel  computers.  The  exploitation  techniques 
vary  depending  on  the  kind  of  parallelism  and  the  parallel  computers  involved.  Ex- 
pressing and  exploiting  parallelism  in  the  software  is  most  important  part  of  software 
development  for  parallel  processing  systems  and  is  the  subject  of  this  study. 

In  the  section,  we  briefly  summarize  the  existing  parallel  computers.  These  can 
be  divided  into  two  categories:  special-purpose  and  general-purpose.  The  special- 
purpose  parallel  computers  are  targeted  to  solve  specific  problems.  The  vector  pro- 
cessors in  super  computers,  such  as  Fujitsu  VP-200  and  Hitachi  S-8101,  have  been 
developed  to  exploit  parallelism  mainly  within  loops.  In  these  computers,  the  vector 
processors  are  controlled  by  a main  control  unit.  Processor  arrays  consist  of  a number 
of  identical  processors  under  the  control  of  a common  control  unit.  Each  processor 
has  nror-se  to  its  own  data,  and  thus  the  same  operation  can  be  performed  simul- 
taneously on  many  data  items.  In  Flynn's  classification  [29),  this  kind  of  computer 
belongs  to  SIMD  (Single  Instruction  Stream,  Multiple  Data  Stream).  The  parallel 
computers  belonging  to  this  category  include  Connection  Machine  [39],  DAP  (Dis- 
tributed Array  Processors)  [41]  and  IBM  GF11  (11).  Systolic  arrays  [53]  consist  of 

'These  computers  can  be  classified  as  general-purpose  computers.  However,  because  their  sup- 
port to  parallel  processing  is  limited  only  to  vector  processing,  we  classify  them  as  special-purpose 


two  or  more  dimensions  of  array  processors  in  which  each  processor  has  its  own  con- 
trol to  perform  specific  task.  The  data  stream  passes  through  the  array  processors 
to  produce  a final  result. 

In  comparison  to  those  special-purpose  parallel  computers,  the  general-purpose 
computers  belong  to  MIMD  (Multiple  Instruction  Stream,  Multiple  Data  Stream) 
according  to  Flynn’s  classification.  The  parallel  computers  in  this  category  are  a set 
of  asynchronous  parallel  processors.  These  computers  can  be  further  divided  into 
shared  memory  parallel  computers  and  distributed  memory  computers.  Because  the 
focus  of  this  research  is  to  develop  a parallel  programming  approach  for  general- 
purpose  parallel  processors,  we  will  further  examine  each  of  the  two  kinds  of  parallel 

An  MIMD  shared-memory  parallel  computer  consists  of  a number  of  processors 
all  having  access  to  a single  shared  memory.  The  processors  communicate  by  read 
and  write  operations,  and  are  connected  to  the  shared  memory  via  one  or  more 
shared-buses  or  interconnection  networks.  As  the  number  of  processors  in  the  system 
increases,  the  communication  medium  becomes  a bottleneck  in  terms  of  performance 
as  well  as  cost.  Thus,  a linear  speed-up  with  an  increase  in  the  number  of  processors 
is  not  achievable  or  is  limited  to  a certain  number  of  processors.  In  the  MIMD 
shared-memory  parallel  computers,  the  major  software  design  problems  include  data 
access  synchronization  and  load  balancing.  The  shared-memory  parallel  computers 
include  Cray  X-MP  and  Y-MP  series  (45),  Alliant  FX8  [76],  Encore  Multimax  [45], 
IBM  RP3  [66]  and  Sequent  Balance  [77]. 

To  model  the  computation  on  the  shared-memory  parallel  computers,  a theoreti- 
cal model  of  parallel  computation,  called  PRAM  (Parallel  Random  Access  Memory) 
[30]  can  be  used.  In  the  PRAM  model,  each  processor  is  a RAM  (Random  Access 
Machine).  Processors  communicate  by  reading  from  and  writing  to  a global  memory. 


The  instructions  in  each  processor  are  executed  in  a synch 
were  a global  clock.  PRAM  models  can  be  further  divided  according  to  the  resolu- 
tion schemes  for  the  simultaneous  read  and  write  access  - Exclusive  Read  Exclusive 
Write  (EREW),  Concurrent  Read  Exclusive  Write  (CREW),  Exclusive  Read  Con- 
current Write  (ERCW)  and  Concurrent  Read  Concurrent  Write  (CRCW).  Although 
the  underlying  assumptions  in  such  PRAM  models,  such  as  constant  communication 
lime  and  synchronous  execution  of  instruction,  are  not  realistic,  they  have  been  used 
as  the  multiprocessor  representations  in  the  evaluation  of  parallel  algorithms  and  task 
scheduling  problems. 

An  MIMD  distributed  parallel  computer  consists  of  a set  of  processors  each  hav- 
ing a non-shared  local  memory.  Processors  communicate  by  message  passing  via 
communication  channels.  In  the  MIMD  distributed  memory  parallel  computers,  the 
synchronisation  is  implicit  through  communication.  The  popular  interconnection 
networks  include  hyptreube,  ring,  tree  and  mesh.  Unlike  shared  memory  parallel 
computers,  distributed  memory  parallel  computers  are  not  affected  by  the  memory 
contention  problem  and  are  more  easily  expandable.  One  of  the  problems  encoun- 
tered by  the  distributed  memory  parallel  computers  is  a message  passing  latency  due 
to  the  possible  transferring  of  data  via  intermediate  processors.  The  major  software 
design  problems  include  data  placement,  communication  overhead  and  scheduling. 
The  shared  memory  parallel  computers  can  be  seen  as  a special  form  of  distributed 
memory  parallel  computers  in  which  all  the  processors  are  fully  connected.  Our  soft- 
ware development  approach  is  targeted  for  the  MIMD  distributed  memory  parallel 
computers.  Any  approach  for  the  MIMD  distributed  memory  parallel  computers 
can  be  adopted  to  MIMD  shared  memory  parallel  computers  with  minor  modifica- 
tions. The  MIMD  distributed  memory  parallel  computers  include  hypercube  [35], 
NCUBE  [64],  BBN  [16]  and  Inmos  Transputer  network  [56]. 


In  this  dissertation,  a software  development  approach  for  parallel  processing  sys- 
tems is  presented.  The  approach  is  based  on  an  integrated  object-oriented  and 
functional  paradigm.  We  develop  a computation  model  PROOF  (PaRalicl  Object- 
Oriented  Functional)  based  on  object-oriented  and  functional  paradigms  (84].  This 
PROOF  computation  model  permits  the  expression  of  explicit  and  implicit  paral- 
lelism using  objects  and  functions  in  various  levels  of  granularity.  This  computa- 
tion model  serves  as  a platform  in  developing  our  approach  to  programming  parallel 
processors.  It  provides  a framework  for  programmers  to  express  parallelism  in  the 
application.  Once  parallelism  is  expressed  using  PROOF,  it  can  be  exploited  when 
the  PROOF  program  is  transformed  to  a target  code.  The  transformation  is  done  in 
two  steps.  The  first  step  transforms  the  PROOF  program  to  the  IPR  which  is  inde- 
pendent of  both  a target  language  and  a target  machine.  We  do  analysis  on  the  IPR 
to  determine  the  proper  grain  sizes.  The  second  step  transforms  the  IPR  to  a target 
code,  using  the  information  generated  by  the  grain  size  analysis.  The  dissertation  is 
organized  as  follows. 

Chapter  2 surveys  computation  models  and  software  development  approaches 
for  parallel  processing  systems’  Three  parallel  programming  approaches-parallelizing 
compiler  approach,  programming  language  constructs  approach  and  parallel  program- 
ming language  approach-are  discussed.  In  particular,  the  existing  parallel  program- 
ming language  approaches  have  been  extensively  discussed.  Chapter  3 presents  the 
PROOF  computation  model.  In  this  chapter,  the  characteristics  of  the  PROOF 
are  presented  along  with  examples.  Major  characteristics  includes  incorporation  of 
object-oriented  concepts  to  functional  paradigm,  allowing  various  granularity  levels 


’We  have  shortened  the  phr. 
gramming  parallel  processors'. 


of  parallelism,  integration  of  referential  transparency  with  history  sensitivity  and  in- 
tegration of  parallelism  with  inheritance.  Chapter  4 presents  the  IPR  and  the  rules 
for  two-phase  transformation  of  the  PROOF/L1  programs.  A PROOF  program  is 
first  transformed  to  the  IPR  code,  and  then  the  IPR  code  is  transformed  to  a target 
code.  We  show  that  such  a transformation  preserves  the  correctness  of  the  original 
program.  Chapter  5 presents  an  allocation  strategy  for  exploiting  parallelism  in  pro- 
grams based  on  the  computation  model  PROOF.  We  develop  grain  sire  determination 
algorithms  on  various  types  of  parallelism.  In  the  case  of  pipe-lined  parallelism,  we 
show  that  our  algorithm  can  find  optimal  solutions.  In  the  cases  of  tree  parallelism 
and  graph  parallelism,  we  introduce  the  concept  of  gain  to  analyze  grain  sizes.  We 
also  show  that  our  approach  performs  better  than  the  existing  approaches.  Chapter 
6 summarizes  the  dissertation  and  discuss  the  directions  of  future  research. 


SThc  PROOF/L  is  a prototype  programming  language  based  on  the  computation  model  PROOF. 
The  syntax  of  the  PROOF/L  is  given  in  Appendix  At. 


COMPUTATION  MODELS  AND  PARALLEL  PROGRAMMING 


Id  this  chapter,  we  have  surveyed  the  existing  computation  model  and  their  sup- 
ports to  parallel  programming,  and  the  existing  approaches  to  programming  parallel 
processors.  This  chapter  is  organized  as  follows.  In  Section  2.1,  the  existing  compu- 
tation models  and  its  extension  for  parallel  processing  are  surveyed.  In  Section  2.2, 
the  existing  parallel  programming  approaches  are  compared,  and  the  parallel  lan- 
guage approach  is  identified  as  the  most  promising.  In  Section  2.3,  the  existing 
parallel  programming  approaches  are  surveyed.  In  Section  2.4,  the  contributions  of 
this  dissertation  arc  outlined. 


As  the  various  cost-effective  MIMD  parallel  computers  begin  to  appear  on  the 
market,  more  general  approaches  to  programming  such  computers  are  required  so 
that  the  programmers  can  write  programs  without  concern  about  the  details  of  the 
parallel  computers.  Because  programming  languages  have  been  used  as  communica- 


the  first  computer,  the  programming  languages  in  writing  parallel  programs  play  an 
important  role  in  the  parallel  programming  systems.  More  importantly,  the  underly- 
ing computation  model  on  which  the  semantics  of  the  programming  language  is  based 
has  significant  impact  on  the  parallel  programming  systems.  There  are  many  pro- 
gramming languages  used  in  writing  programs  for  these  parallel  processing  systems. 
Since  each  of  these  programming  languages  has  different  features,  their  comparison 


lion  vehicles  between  the  programmers  and  the  computers  since  the  appearance  of 
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task.  Rather  than  examining  each  of  these  programming  languages 


in  terms  of  the  specific  features,  we  classify  them  into  five  different  categories  - im- 
perative, logic,  functional,  object-oriented  and  knowledge-baaed  - according  to  their 
underlying  computation  model. 


As  Backus  |8)  described  their  programming  style  as  one-word-at-a-time  style,  the 
programming  languages  in  this  category  have  been  significantly  influenced  by  the  von 
Neumann  architecture.  In  imperative  programming,  computation  is  achieved  by  the 
repeated  and  step-by-step  computation  of  low-level  values  and  the  assignment  of  these 
values  to  memory  locations.  Programmers  explicitly  specify  the  order  of  instruction 
execution  using  control  structures.  This  is  also  called  as  the  von  Neumann  model  of 
programming. 

There  have  been  ways  to  use  the  programming  languages  in  this  model  for  parallel 
processing.  In  one,  the  programmers  write  sequential  programs  and  let  parallelising 
compilers  delect  parallelism  in  the  programs.  This  has  been  widely  used  primarily 
because  the  programmers  need  not  make  additional  effort  in  making  their  programs 
to  execute  in  parallel.  However,  those  compilers  can  only  detect  localized  parallelism 
within  loops. 

Another  way  is  to  extend  existing  imperative  languages  with  parallel  language 
constructs.  Since  the  parallel  constructs  are  added  to  inherently  sequential  program- 
ming languages,  the  programmers  are  heavily  burdened  with  explicitly  specifying  par- 
allelism, communication  and  synchronization  among  parallel  units.  Fortran  [3,  20], 
C [5,  28],  Ada  [22],  and  Occam  |46]  belong  to  this  category. 
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In  logic  programming,  the  basic  element  is  the  clause.  A logic  program  consists  of 
a set  of  clauses  - goal  clause,  assertion  clause  and  conditional  clauses  - declaring  the 
goal,  fact  and  properties  that  describe  the  problem  for  which  the  solution  is  sought. 
The  implementation  system(interpreter),  then,  uses  a unification  process  to  show  if 
the  goal  clause  is  true  or  not.  One  goal  may  consist  of  a number  of  sub-goals  and 
only  be  solved  by  solving  all  sub-goals  independently.  Another  goal  may  be  solved  by 
solving  only  some  of  sub-goals.  This  gives  rise  to  a great  deal  of  potential  parallelism: 
AND-parallelism,  OR-parallelism  and  Stream-parallelism. 

Efficiency  of  execution  is  a major  problem  in  this  model  of  programming.  Thus, 
even  if  a logic  programming  language  might  be  suitable  for  a certain  application,  lack 
of  efficiency  may  restrict  its  use  primarily  to  a rapid  prototyping  tool.  The  program- 
ming languages  belonging  to  this  category  include  PARLOG  (19]  and  Concurrent 
PROLOG  [71]. 

2.1.3  Functional  Computation  Model 

In  imperative  programming,  the  basic  element  of  programming  is  the  assign- 
ment statement,  which  modifies  a value  of  a variable  as  a result  of  computation.  In 
functional  programming,  the  basic  clement  is  a function,  which  receives  input  and 
produces  output  as  a result  of  function  application. 

A function  describes  the  relationship  between  input  and  output.  The  essence  of 
the  functional  computation  is  to  combine  such  functions  and  produce  more  powerful 
functions  so  that  a solution  for  a problem  can  be  obtained  by  applying  such  functions 
to  input.  The  functional  languages  based  on  this  mode)  are  referentially  transparent 
and  the  programmers  cannot  introduce  race  conditions.  As  a result,  this  model  has 
great  potential  of  exploiting  implicit  parallelism  by  removing  side-effects  caused  by 


assignment  statements.  Functional  programming  has  been  one  of  the  main  directions 
in  developing  new  languages  that  directly  address  the  challenge  of  parallel  programs. 
In  functional  languages,  computational  abstractions  arc  expressed  through  functions. 
A first-order  function  takes  data  objects  as  arguments  and  produces  new  data  objects 
as  results.  The  function  abstracts  the  method  used  to  produce  the  new  objects  from 
the  arguments.  High-order  functions  generalize  this  further  by  taking  data  objects 
as  well  as  other  functions  as  arguments  and  producing  new  data  objects  and  new 
functions.  It  can  be  further  advanced  by  the  referential  transparency.  Because  the 
arguments  of  a function  could  be  evaluated  in  any  order,  functional  programming  lan- 
guages promise  to  be  more  portable  and  easily  maintained  through  an  entire  software 

However,  due  to  its  history  insensitivity,  the  expressive  power  of  programming 
is  limited.  This  model  is  not  suitable  for  expressing  inherently  concurrent  natures. 
The  data  flow  model  shares  many  common  properties  with  the  functional  model.  In 
the  data  flow  model,  data  “flows"  from  one  statement  to  another,  execution  of  state- 
ments is  data  driven,  and  identifiers  obey  the  single-assignment  rule.  In  a data  flow 
computer,  the  availability  of  input  operands  triggers  the  execution  of  the  instruction 
which  consumes  the  inputs.  The  programming  languages  belonging  to  this  category 
include  ParAlfl  |42),  SISAL  [60]  and  Multilisp  [38]. 


The  object-object  programming  model  is  based  on  the  concept  of  an  object.  A 
program  is  a set  of  objects.  Each  object  is  a self-contained  entity  with  its  own  private 
data  and  a set  of  methods  to  manipulate  those  data.  Any  access  to  the  data  of  an 
object  must  be  done  by  calling  an  appropriate  method.  The  behavior  of  an  object 
is  defined  by  its  class,  which  is  a description  of  a set  of  objects  having  a common 
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behavior.  An  inheritance  mechanism  allows  a class  to  be  defined  as  an  extension  of 
another  class. 

In  a traditional  object-oriented  programming  model,  there  is  always  one  active 
object  at  a time.  The  active  object  can  send  a message  to  a receiver  object.  Then  the 
receiver  object  becomes  active  and  the  sender  object  must  wait.  After  the  receiver 
object  returns  a result  to  the  sender  object  and  becomes  inactive,  the  sender  object 
can  continue  its  computation.  To  incorporate  parallelism  in  this  model,  a number  of 
means  have  been  proposed  (9): 

1)  allow  an  object  to  be  active  without  receiving  messages. 

2)  allow  the  receiving  object  to  continue  execution  after  it  returns  its  result. 

3)  broadcast  a message  to  several  objects  at  once. 

4)  allow  the  sender  of  a message  to  proceed  in  parallel  with  the  receiver. 

Any  combination  of  these  can  be  used. 

An  object-oriented  programming  model  naturally  reveals  existing  parallelism  in 
problems  [83].  Besides  the  advantages  of  modifiability,  maintainability  and  reusabil- 
ity, one  advantage  of  this  model  over  others  is  that  the  concept  of  an  object  can 
be  used  at  earlier  stages  of  software  development  cycles  than  the  implementation 
stage.  It  implies  that  parallel  processing  aspects  such  as  parallelism  and  communica- 
tion among  parallel  components  can  be  naturally  handled  at  the  earlier  stage  of  the 
software  development  development.  Consequently,  it  is  easy  for  the  programmers  to 
handle  parallelism  and  communication  among  parallel  components.  However,  in  this 
model,  parallel  execution  among  concurrent  objects  is  the  only  source  of  parallelism, 
and  the  amount  of  parallelism  to  be  exploited  may  not  be  sufficient  for  effectively 
utilizing  many  fine-grain  processors.  The  programming  languages  belonging  to  this 
category  include  ABCL/1  [89],  Act  1 [54],  Concurrent  Smalltalk  [88],  Emerald  [12] 
Mental  [36]  and  POOL  [4]. 


In  addition  to  those  computation  models  we  presented,  there  are  at  least  three 
computation  models  which  perform  their  computation  in  different  ways  from  the 
previous  models.  The  rule-based  model  performs  the  computation  by  firing  rules.  In 
a production  system  language,  such  as  OPS5  [14],  the  basic  concepts  are  IF-THEN 
statements.  These  rules  are  repeatedly  executed  until  none  of  IF  conditions  is  true.  In 
cellular  models,  the  computation  is  performed  by  a set  of  cells.  Each  cell  performs  its 
computation  by  communicating  with  its  neighbors  in  a regular  pattern  that  matches 
the  flow  of  data  and  control  in  the  target  computation.  The  class  of  programming 
language  belonging  to  this  model  is  a semantic  network  language,  such  as  NETL  [27]. 
In  neural  models,  the  basic  concepts  are  statements  defining  the  network  topology 
together  with  the  recall  (and  training)  functions  of  the  neurons.  The  execution  occurs 
each  time  the  network  is  updated.  The  languages  belonging  to  this  group  can  be 
regarded  as  knowledge- based  languages  and  are  used  for  applications  in  artificial 
intelligence,  cognitive  psychology,  and  learning  systems. 

2.2  Overview  of  Parallel  Programming  Approaches 
There  arc  three  approaches  to  programming  parallel  computers.  One  approach 
is  to  write  programs  using  conventional  sequential  programming  languages,  such  as 
Fortran  [5, 20, 43]  or  C [5, 28],  and  parallelize  the  programs  using  parallelizing  or  vec- 
torizing compilers.  Although  this  approach  seems  to  be  attractive  since  many  types  of 
existing  sequential  software  can  be  adapted  to  such  a parallel  programming  environ- 
ment with  minor  modifications,  it  is  hardly  an  effective  approach.  The  parallelizing 
or  vectorizing  compilers  fail  to  unravel  most  of  the  parallelism.  They  can  only  detect 
parallelism  associated  with  iterations  over  common  data  structures,  such  as  arrays 
and  matrices  [7],  and  require  extensive  dependency  analysis  [3].  Usually,  sequential 


languages  are  extended  with  compiler  directives  in  order  to  help  the  compilers  detect 
parallelism.  Because  these  extensions  are  machine-specific,  portability  of  programs 
is  hampered.  Even  after  extensive  research  and  development,  parallelizing  compil- 
ers have  not  met  the  expectations  that  were  raised  for  them.  The  problem  in  this 
approach  is  not  the  compilers  themselves,  but  the  inherent  sequential  characteristics 
of  imperative  programming  languages.  As  we  discussed  above,  those  languages  are 
designed  for  sequential  execution  in  sequential  processors.  Although  this  approach  is 
very  popular  in  scientific  application  areas,  it  docs  not  provide  promising  directions 
for  the  future  of  parallel  programming. 

Another  approach  is  to  use  parallel  language  constructs  to  explicitly  model  the 
parallelism  in  programs.  These  parallel  language  constructsincludetheparallelstate- 
ment  and  input,  output  commands  in  CSP  (40],  monitor  and  wait,  signal  operations  in 
Concurrent  Pascal  [13],  and  task  and  rendezvous  mechanisms  in  Ada  (22].  Although  in 
this  approach  the  imperative  languages  are  extended  with  some  language  constructs, 
the  basic  model  of  computation  is  still  sequential  as  in  the  first  approach.  Using 
the  parallel  language  constructs,  it  is  the  programmers’  responsibility  to  explicitly 
express  the  parallelism  and  ensure  the  correct  communication  and  synchronization 
among  parallel  units,  which  is  an  extremely  complex  and  difficult  task.  This  is  one 
of  the  major  haws  in  software  systems  for  parallel  computing,  and  there  is  no  easy 
solution  in  sight.  In  addition,  these  parallel  language  constructs  are  only  suitable 
to  express  coarse  grain  parallelism.  Thus,  massive  and  fine  parallelism  cannot  be 
expressed  in  this  approach. 
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The  third  approach  is  to  use  parallel  programming  languages,  such  as  Id  Nouveau 
[62],  and  SISAL  [60],  which  arc  functional  languages  tailored  for  scientific  computa- 
tion, PAKLOG  [19],  a parallel  logic  language,  and  Act  1 [54],  an  object-based  lan- 
guage based  on  the  Actor  model.  The  underlying  computation  models  of  these  paral- 
lel programming  languages  arc  fundamentally  different  from  the  underlying  models  of 
imperative  programming  languages  in  that  parallelism  is  mostly  implicit  and  massive 
parallelism  is  easily  obtainable.  Hence,  the  programmers  using  these  languages  are 
liberated  from  the  complications  caused  by  parallelism.  This  is  considered  the  most 
promising  approach  to  parallel  programming,  and  our  computation  model  belongs  to 
this  category.  We  will  discuss  it  further  in  the  following  section. 

2.3  Parallel  Language  Approaches 

Sarkar  [70]  has  identified  three  fundamental  issues  to  be  resolved  for  parallel 
execution  of  a program:  1)  Identifying  parallelism  in  the  program,  2)  partitioning  the 
program  into  sequential  tasks,  and  3)  scheduling  the  tasks  on  processors.  Although 
these  issues  have  not  been  treated  as  a whole,  they  are  closely  related  in  the  sense  that 
the  choice  of  a strategy  in  one  area  may  significantly  affect  the  choice  of  a strategy  in 
other  areas.  Sarkar  introduced  a compile  time  method  for  automatically  partitioning 
data  flow  graphs.  The  goal  is  to  find  a schedule  to  minimise  the  completion  time  of  the 
program  by  avoiding  exploitation  of  too  fine  a grain  of  parallelism.  This  approach 
incorporates  three  steps:  1)  construct  the  program  graph,  2)  partition  the  graph 
and  3)  generate  the  target  code.  In  step  1 ),  the  execution  time  is  assigned  to  each 
node  and  the  communication  time  between  two  neighboring  nodes  is  also  assigned  to 
the  edge  connecting  the  two  nodes.  In  step  2),  the  program  graphs  are  partitioned. 
Finally,  in  step  3),  the  macro  data  flow  modules  arc  generated.  Each  macro  data  flow 
module  is  a code  segment  to  be  executed  sequentially  in  one  processor.  This  method 


has  been  applied  lo  the  SISAL  programs,  and  is  regarded  as  the  most  sophisticated 
methods  to  partition  a program.  However,  this  automatic  partitioning  system  can 
only  exploit  parallelism  appearing  in  the  program  graph.  Some  patterns  of  parallelism 
cannot  be  shown  in  the  program  graph,  and  thus  the  compiler  usually  does  not  have 
enough  information  to  detect  such  parallelism  and  exploit  it  in  a proper  way.  For 
instance,  pipelined  parallelism  is  almost  impossible  to  exploit  in  this  approach  since 
the  compiler  itself  cannot  detect  pipelined  parallelism.  In  addition,  the  parallelism 
due  to  the  arbitrary  producer-consumer  relationship  cannot  be  exploited  at  all. 

Goldberg  [34)  developed  a method  lo  programming  parallel  processors  for  func- 
tional programs  by  introducing  a logical  construct  called  a serial  combinator.  A serial 
combinator  is  defined  as  a function  with  the  following  properties:  1)  its  body  contains 
no  free  variables;  2)  its  body  is  sequential  and  contains  constructs  for  synchronising 
its  execution  with  other  tasks;  3)  its  body  could  not  occur  as  a subexpression  within 
the  body  of  another  serial  combinator.  In  this  approach,  the  third  property  implies 
that  the  programmers  have  to  determine  as  few  serial  combinators  as  possible,  since 
they  cannot  be  coalesced  to  form  a bigger  serial  combinator.  It  also  implies  that 
the  program  developed  for  one  parallel  computer  may  not  be  directly  portable  due 
to  possible  performance  degradation.  In  addition,  the  programmers  must  ensure  the 
correct  synchronisation  and  communication  among  tasks. 

In  [47,  48),  Jagannathan  et  al.  introduced  an  approach  to  programming  a net- 
work of  workstations  based  on  a coarse-grain  dataflow  concept.  Their  computation 
model  incorporates  two  different  paradigms:  at  the  lower  level  procedural  language 
is  used  to  express  functions  or  procedures;  at  the  high  level  a declarative  style  of 
programming  is  used  to  express  implicit  parallelism  among  the  functions  written  in 
procedural  programming  languages.  Foster  and  Taylor  [31]  introduced  Strand  for 


parallel  programming,  which  is  based  on  logic  computation  model.  Strand  can  pro- 
vide an  interface  to  other  languages  as  in  |47,  48].  They  have  ignored  the  issue  of 
the  granularity  of  parallelism  and  assume  that  the  programmers  will  make  choices  on 
the  grain  size  during  the  development  of  the  application.  Another  approach  based 
on  the  coarsc-grain  dataflow  computation  model  was  introduced  by  Gaudiot  and  Lee 
[33].  Their  approach  was  aimed  at  the  use  of  shared  memory  parallel  processors  for 
solving  scientific  problems. 

Although  Linda  [15]  is  not  based  on  any  specific  compulation  model  or  approach, 


it  is  worth  comparing  it  with  other  approaches.  Linda  is  a small  set  of  operations  that 
can  be  added  to  a base  language  to  create  a parallel  processing  dialect.  The  concept 
of  Linda  is  based  on  the  tuple  space  of  parallel  processing.  Processes  and  data  can  be 
considered  to  be  elements  in  tuple  space.  Communication  between  processes  occurs  in 
the  following  way:  the  sender  creates  data  in  tuple  space;  the  receiver  gets  the  data  in 
the  tuple  space,  hence  communication  takes  place.  Linda  provides  the  following  four 
basic  operations:  in,  out,  rtf  and  eval.  in  removes  the  tuple  read  from  the  tuple  space, 
rrf  reads  the  tuple,  but  leaves  the  tuple  to  be  read  by  other  processes,  ouf  creates  a 
new  tuple  and  places  it  in  the  tuple  space,  and  evat  creates  a new  tuple  by  generating 
a process.  A disadvantage  of  Linda  is  that  the  programmers  have  to  write  programs 
in  terms  of  communication  with  other  processes.  In  addition,  its  implementation 


communication  via  shared  memory.  Another  disadvantage  in  Linda  is  that  the  use  of 
conceptually  shared  memory  does  not  provide  any  protection  from  erroneous  access 
to  any  data  item  in  the  shared  memory.  When  we  consider  the  fact  that  data  or 
messages  need  not  be  accessible  to  any  other  processes  than  those  processes  which 
need  them,  Linda  cannot  support  an  information  hiding  principle. 


In  conclusion,  these  existing  approaches  share  some  or  all  of  the  following  disad- 
vantages: 

• Software  engineering  concepts  for  managing  parallelism  have  not  been  fully 

• Most  approaches  arc  targeted  for  the  shared  memory  processors. 

• The  concept  of  shared  data  has  not  been  introduced  into  programming  parallel 


• Coding  of  correct  synchronization  and  communication  using  explicit  constructs 
is  still  the  programmer’s  responsibility. 

In  order  to  develop  software  for  parallel  processing  systems,  much  effort  is  still  needed 
to  address  these  problems. 

2.4  Contributions  of  the  Dissertation 

In  developing  an  approach  to  software  development  for  parallel  processing  sys- 
tems integrating  object-oriented  and  functional  paradigms,  the  dissertation  makes 
the  following  contributions: 

• The  Parallel  Object-Oriented  Functional  Computation  Model  (PROOF)  has 
been  developed.  In  PROOF,  the  object-oriented  concepts  have  been  inte- 
grated to  the  functional  paradigm  without  sacrificing  the  advantages  of  either 
paradigm.  PROOF  offers  the  following  advantages:  1)  Parallelism  in  differ- 
ent granularity  levels  can  be  exploited.  2)  By  separating  synchronization  con- 
straints from  the  operation  of  the  method,  inheritance  and  parallelism  can 
be  integrated  without  interference.  3)  Parallel  aspects  of  the  software  can  be 
treated  as  a primary  issue  at  the  early  stage  of  the  software  development  so  that 
the  complications  at  the  later  phases  of  the  software  life  cycle  can  be  reduced. 
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The  intermediate  form,  called  Intermediate  Program  Representation  (IPR),  for 
the  internal  representation  of  the  PROOF/L  programs  has  been  developed. 
The  IPR  is  a hybrid  graphical  representation  in  which  object-level  behavior  is 
represented  as  a Petri-net  and  method-level  behavior  is  represented  as  function 
nodes  and  their  data  dependency  relationships.  Correctness  preserving  trans- 
formation rules  from  the  PROOF/L  program  to  the  IPR  and  from  the  IPR  to 
a target  language  have  also  been  developed.  The  two-level  transformation  via 
the  IPR  facilitates  the  separation  of  the  semantic  issues  from  the  performance- 
oriented  issues. 

A two  level  allocation  approach  has  been  developed.  At  the  object-level,  we 
model  the  PROOF  program  as  a directed  graph  by  analyzing  the  body  of 
each  object  and  then  duster  the  objects  so  that  each  cluster  can  be  analyzed 
separately  at  the  method-level  analysis.  At  the  method-level,  grain  size  deter- 
mination techniques  for  various  types  of  parallelism  have  been  developed  and 
compared  with  other  existing  approaches  using  examples.  In  order  to  analyze 
the  grain  sizes,  we  develop  the  notion  of  gain  with  which  the  possible  contribu- 
tion to  the  reduction  of  the  completion  time  of  the  program  can  be  determined. 


CHAPTER  3 

PROOF  COMPUTATION  MODEL 

The  object-oriented  paradigm  is  considered  promising  for  the  development  of 
software  for  parallel  processing  systems  [83,  85,  86, 87].  Such  a paradigm  is  based  on 
the  principle  of  information  hiding.  The  main  mechanisms  associated  with  the  object- 
oriented  paradigm  are  data  encapsulation  and  inheritance.  These  mechanisms  offer 
many  advantages,  such  as  comprehensibility,  reusability,  maintainability,  modularity 
and  extensibility.  In  addition,  the  object-oriented  paradigm  can  naturally  reflect  the 
structure  of  the  problem  space.  In  traditional  object-oriented  computation  models, 
there  is  always  one  active  object  at  a time.  The  active  object  can  send  a message  to  a 
receiver  object.  Then  the  receiver  object  becomes  active  and  the  sender  object  waits. 
After  the  receiver  object  returns  a result  back  to  the  sender  object  and  becomes 
inactive,  the  sender  object  can  continue  its  computation. 

This  model  is  inherently  sequential  since  it  was  developed  for  sequential  comput- 
ers. Some  attempts  have  been  made  to  incorporate  parallelism  in  the  object-oriented 
paradigm.  In  POOL-T  [4],  parallelism  is  achieved  by  allowing  execution  of  the  objects 
without  receiving  a message.  Concurrent  Smalltalk  [88],  Hybrid  [61]  and  languages 
based  on  the  Actor  model  [2],  such  as  Act++  [49]  and  Rosette  [78],  introduce  paral- 
lelism by  adopting  asynchronous  message  passing.  The  actor  model  supports  delega- 
tion instead  of  inheritance.  However,  in  these  languages,  only  parallelism  among  the 
objects  can  be  expressed  and  thus  exploitation  of  fine  grain  parallelism  is  difficult  or 
expensive.  Our  computation  model  is  an  approach  for  exploitation  of  various  levels 
of  granularity  of  parallelism  based  on  object-oriented  and  functional  paradigms. 
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In  PROOF,  a program  is  represented  as  a set  of  the  objects  which  can  be  executed 
in  parallel.  Each  object  is  an  instance  of  a class,  and  each  has  its  own  local  data 
and  methods.  The  methods  in  PROOF  are  defined  as  purely  applicative  functions. 
Parallelism  at  different  levels  of  granularity  can  be  exploited.  The  major  features  of 
PROOF  are: 

• Class,  object,  and  inheritance  are  supported  in  PROOF  with  the  restriction 
that  all  the  methods  in  the  objects  are  applicative  functions.  In  other  words,  a 
functional  paradigm  is  adopted  at  the  level  of  method  definition. 

• The  guard  of  the  method  is  introduced  to  support  the  synchronization  between 
the  concurrent  objects.  Furthermore,  it  will  be  shown  that  the  methods  along 
with  their  guards  are  inheritable. 

• Objects  are  persistent  in  PROOF.  The  reception  of  values  by  the  objects  is 
introduced  to  modify  the  objects. 

These  features  are  discussed  in  detail  in  the  following.  This  chapter  is  organized  as 
follows.  In  Sections  3.1-  3.5,  the  features  of  PROOF  are  presented  with  examples. 
In  Section  3.6,  the  sources  of  parallelism  in  PROOF  and  its  semantics  are  presented. 

3,1 . Chwts  and  Object; 

Class  interface  and  definition 

In  PROOF,  a program  consists  of  a set  of  the  objects.  Every  object  is  an  instance 
of  a class.  A class  is  a template  for  a set  of  the  objects  bearing  similar  behavior, 
and  it  is  defined  as  a generic  abstract  data  type.  A class  is  defined  by  its  interface 

which  are  purely  applicative  functions. 


22 


class  interface  Bounded -Buffer  (iterntype,  size) 

method  put  ::  Bounded-Buffer  - itemtype  - Bounded-Buffer 
method  get ::  Bounded-Buffer  -*  Bounded Jufferx  itemtype 
method  is.empty  ::  Bounded  buffer  -•  bool 
method  length  ::  Bounded-Buffer  -•  int 
end  class 

Figure  3.1.  The  interface  of  the  class  Bounded-Buffer. 

It  is  important  to  point  out  that  restricting  the  methods  of  classes  applicative 
functions  docs  not  restrict  the  expressive  power  of  PROOF,  but  merely  requires  that 
all  the  effects  of  the  methods  be  explicitly  specified  via  the  parameters  of  the  methods. 
Example  1.  Class  interface  and  definition  of  Bounded-Buffer. 

We  introduce  the  Bounded  .Buffer  example  to  illustrate  the  features  of  PROOF.  The 
class  Bounded-Buffer  is  a FIFO  queue  of  a limited  capacity.  The  class  interface  and 
the  definition  of  Bounded  -Buff  er  are  shown  in  Figures  3.1,  and  3.2.  Bounded-Buffer 
has  two  parameters:  itemtype  and  size,  itemtype  indicates  the  type  of  elements 
to  be  stored  in  the  buffer  and  size  specifies  the  capacity  of  the  buffer.  The  method 
names  are  self-explanatory.  In  the  class  definition,  the  composition  clause  defines 
the  composition  of  the  local  data  of  the  class.  The  composition  of  Bounded  .Buffer 
is  a Cartesian  product.  The  components  can  be  referred  using  the  dot  syntax,  such 


Inheritance  and  genericity 

Both  inheritance  and  genericity  are  supported  in  PROOF.  Classes  in  PROOF 
are  related  by  inheritance  relations.  Inheritance  is  used  to  define  a subclass  as  a 
specialization  of  a superclass.  In  a subclass,  all  the  local  data  and  the  methods  of  its 
superclass  are  inherited.  Additional  local  data  and  new  methods  may  be  introduced. 
The  inherited  methods  may  also  be  overridden  by  a new  definition  of  the  method. 


24 


class  interface  Extended-Buffer(itemtype,  size) 

inherit  put,  get,  is-empty,  length  from  Bounded-Buffer 
method  pop  ::  Extended-Buffer  — Extended-Buffer  x itemtype 
end  class 

class  definition  Extended-Buffer  (itemtype,  size) 
superclass:  Bounded-Buffer  (itemtype,  size) 
inherit:  put,  get,  is-empty,  length 
method  pop  b 
expression 

C /JCfront.dec]  b,  last(b. store)  ] 

end  class 


Figure  3.3.  The  definition  of  tile  class  Extended-Buffer 
Bounded-Buffer  in  Example  1 is  a generic  class.  An  instantiation  of  Bounded-Buffer 
is  the  following: 

class  My-Buffer  instance  of  Bounded_Buffer(my-item,  my-size) 

Class  My-Buffer  is  an  instantiation  of  the  generic  class  Bounded-Buff  er. 

Active  and  passive  objects 

A program  in  PROOF  consists  of  a set  of  the  objects.  The  objects  can  be  classified 
into  the  following  three  categories:  acti ve,  passive  and  pseudo-active. 

Definition  3.1. 1 The  active  object  is  on  object  which  con  initiate  a process  by  invoking 
a method  or  methods  without  any  request  from  the  other  object. 

Definition  S.t.t  The  passive  object  is  an  object  which  has  methods  to  be  invoked  by 
the  other  objects  and  cannot  invoke  them  without  a request  from  other  objects. 

Definition  S.I.S  The  pseudo-active  object  is  an  object  which  cannot  initiate  a process 
but  can  invoke  the  methods  defined  in  another  object  upon  request  by  other  objects. 


class  interface  Producer-Class  (itemtype) 
method  produce  ::  — * itemtype 
end  class 

class  interface  Consumer-Class(itemtype) 
method  consume  itemtype  — * 
end  class 

Figure  3.4.  Interfaces  of  the  classes  Consumer-Class  and  ProducerXlass 
Definition  S.t.l  The  non-passive  object  is  an  object  which  is  not  passive,  and  thus 
includes  active  and  pseudo-active  objects. 

Definition  3A.fi  The  non-active  object  is  an  object  which  is  not  active , and  thus  in- 
cludes pseudo-active  and  passive  objects. 

A non-active  object  acts  like  a service  agency.  It  waits  passively  until  one  of 
its  methods  is  invoked  by  other  objects.  The  non-active  object  may  in  turn  invoke 
methods  in  other  objects.  An  active  object  is  active  initially,  and  it  may  remain 
active  throughout  the  execution  except  for  occasional  suspensions  for  the  purpose 
of  synchronization  with  other  objects.  A body  will  be  attached  to  each  non-passive 
object.  The  bodies  of  the  objects  are  functions  that  may  be  recursive  and  diverse 
(non-terminating). 

Example  3.  Producer-consumer  problem. 

Interfaces  of  two  classes,  Consumer-Class  and  Producer-Class,  are  shown  in 
Figure  3.4.  The  method  produceQ  is  to  generate  an  item  every  time  it  is  called,  and 
the  method  consumeQ  is  to  consume  an  item  every  time  it  is  called.  The  objects 
in  the  producer-consumer  problem  are  shown  in  Figure  3.5.  Producer  and  Consumer 
are  active  objects.  Producer  will  continuously  generate  items  and  Consumer  will 


Object  Buffer:  instance  of  Bounded-Buffer  (my-item,  my-size) 


Active  Object  Producer:  instance  of  ProducerXlass  (ay-item) 

Body  .../*  a body  is  attached  to  Producer*/ 

Active  Object  Consumer:  instance  of  ConsumerjClass  (my-item) 

Body  . . . /*  a body  is  attached  to  Consumer*/ 

Figure  3.5.  The  set  of  the  objects  for  a producer-consumer  problem, 
continuously  consume  items.  The  definition  of  the  bodies  of  Producer  and  Consumer 
will  be  given  later.  Buffer  is  a passive  object,  and  it  becomes  active  when  its  methods 
are  invoked  by  Producer  or  Consumer. 

3.2  Method  Definition 

Methods  in  PROOF  are  purely  applicative  functions  or  functional  forms,  i.e. 
high  order  functions.  We  use  a constructor  [*i, to  denote  a sequence 
of  homogeneous  or  heterogeneous  elements.  In  the  case  of  homogeneous  elements,  it 
denotes  a list  or  an  array  whose  types  are  T'  and  T"(s  T x T x . . . x T)  respectively. 
In  the  case  of  heterogeneous  elements,  it  denotes  a Cartesian  product  whose  type  is 
nr=,  Ti  (=Ti  xT,  x ...  X T„). 

PROOF  supports  a set  of  primary  functions,  such  as  arithmetic  operations,  logic 
operations  and  list-handling  operations  and  functional  forms  from  which  other  func- 
tions and  functional  forms  can  be  easily  constructed.  The  following  are  the  functional 
forms  currently  supported  in  PROOF, 
a)  Functional  form:  a (called  apply  to  alt) 

*»I 

s !/(*.), /(*s) /(*.)] 


o has  two  parameters,  a function  of  type  Ti  — * 7'?)  and  a list  of  homogeneous  elements 
of  type  T\.  The  function  / is  applied  to  each  element  in  the  list  and  yields  a list  of 
elements  of  type  7s. 

b)  Functional  form:  0 (called  distributed  apply ) 

0[fuh All*., *s *.) 

= [/.(*.).  /.(*>) /.(*.)] 

0 has  two  parameters,  a list  of  functions  in  which  each  function  ft  is  of  type  T- 1 * — » 
and  a list  of  heterogeneous  elements  in  which  the  ith  clement  is  of  type 
Each  function  in  the  first  list  is  applied  to  the  corresponding  element  in  the  second 
list.  It  yields  a list  in  which  the  ith  element  is  of  type  T, f11. 

c)  Functional  form  7 (called  filter) 

7fo.  b.  • • • •••,*•] 

([)  if  n = 1 and  &■  = False 

[*1]  if  n = 1 and  b}  = True 

7(6, . . . 64](*1-  • • Xi]  a 7[6lti . . . 6„][iit, ...*„)  if  n > 1 and  2 < k < n 
Here,  0 denotes  the  concatenation  of  two  lists.  7 has  two  parameters,  a list  of  boolean 
and  a list  of  any  elements.  This  function  yields  a subsequence  of  the  second  list  by 
selecting  elements  whose  corresponding  elements  in  the  first  list  are  True, 

d)  Functional  form:  i (called  copied  apply) 

<(/../> AH*] 

= [/.(*).  /.(*),- -,A(x)] 

S has  two  parameters,  a list  of  functions  in  which  each  function  /,  is  of  type  T ->  Tit 
and  an  element  is  of  type  T.  Each  function  in  the  first  list  is  applied  to  the  element 
and  a list  of  elements  having  type  T,  x T,  X ...  T„  is  yielded  as  a result  of  6 application, 

e)  Functional  form:  rj  (called  inserted  apply ) 
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7l/]l*>.*» 

= I/(*i./(*J /(*.-.,/(*.-„*-))•■•)  1 

i)  has  two  parameters,  a function  of  which  type  is  T -+  T and  a list  of  elements  in 
which  each  element  is  of  type  T.  The  function  is  first  applied  to  the  last  two  elements. 
Then  the  result  of  that  application  and  the  third  from  the  end  of  the  list  are  used 
as  input  to  another  function  application.  This  application  is  continued  until  the  first 
element  of  the  list  is  input  to  the  function.  In  fact,  this  functional  form  can  be  given 
a different  meaning  by  allowing  parallel  application  of  the  function  to  the  elements. 
That  is,  we  can  group  the  elements  into  a set  of  pairs  and  apply  the  function  /. 
By  repeating  this  process  until  a value  is  returned  as  a result,  we  can  exploit  tree 

The  following  example  illustrates  the  parallelism  due  to  parallel  evaluation  of 
arguments  and  functions. 

Example  4.  Partition  an  array  of  integers,  A which  has  type  int",  using  a pivot 
element  a e A as  shown  in  Figure  3.6.  In  other  words,  we  want  to  rearrange  the 
elements  in  the  array  A so  that  all  the  elements  whose  values  are  less  than  the  value 
of  s precede  the  elements  whose  values  are  greater  than  or  equal  to  a. 

A solution  to  the  problem  can  be  formulated  as  follows.  First,  we  define  the 
following  two  functions  f and  g: 


One  solution  to  the  problem  is  the  following  one-liner: 
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X < S X>  s 


Figure  3.6.  An  array  partitioning  problem 
(7  (a  f A)  A ) o (7  (o  g A)  A ) 

The  functions  are  curried  and  we  assume  left  association.  This  expression  is  a con- 
catenation of  two  7 functions,  which  can  be  evaluated  in  parallel.  Within  each  of  the 
7 functions,  the  applications  of  f and  g to  A can  also  be  evaluated  in  parallel  owing 
to  the  functional  form  a.  The  execution  of  this  example  is  illustrated  in  Figure  3.7. 

3.3  Synchronisation  of  Objects 

The  methods  defined  in  each  object  may  require  some  preconditions  under  which 
they  can  be  executed.  For  example,  get  in  Bounded  -Buffer  can  be  executed  only 
when  the  buffer  is  not  empty.  When  the  buffer  is  empty,  the  invoker  of  the  method 
must  be  suspended.  This  problem  is  commonly  known  as  a synchronisation  problem. 

In  order  to  specify  the  synchronization  constraints  in  the  parallel  object-oriented 
paradigms,  many  constructs  have  been  proposed.  Among  them  are  critical  sections 
[88],  message  queues  [61],  behavior  abstraction  [49]  and  enabled-sets  [78].  However, 
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Figure  3.7.  A solution  for  the  array  partitioning 


the  use  of  these  constructs  interferes  with  the  inheritance  mechanism,  and  the  syn- 
chronization constraints  are  not  directly  inheritable  with  the  methods.  In  PROOF, 
synchronization  among  the  objects  is  achieved  by  attaching  an  optional  precondition, 
called  guard  [23],  to  each  of  the  methods  in  a class.  Each  guard  is  a predicate.  The 
object  which  invokes  the  method  is  suspended  when  the  attached  guard  evaluates 
to  False,  and  it  is  resumed  when  the  guard  becomes  True.  The  guard  attached  to 
the  method  is  defined  in  a way  that  it  depends  only  on  the  status  of  the  local  data, 
and  does  not  on  the  definition  of  any  other  methods.  Therefore  the  inheritance  of 
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class  definition  Bounded Juff er(itemtype,  size) 

composition  store:list(itemtype)  x count :int 

method  put  b x 

guard (b. count  < size) 
expression 

fil (append .right  x) , inc]  b 
method  get  b 

guard  (b.  count  > 0) 
expression 

[ /)[tail, dec)  b,  head(b. store)  ] 
method  is.empty  b 
expression 

method  length 
expression 

end  class 


Figure  3.8.  The  definition  of  the  class  Bounded-Buffer 
individual  methods  will  not  be  hampered  by  the  inclusion  of  the  guard.  Such  guards 
can  be  inherited  with  the  methods  they  are  attached  to. 

Example  5.  Inheritance  of  synchronization  constraints. 

The  complete  definition  of  the  class  Bounded-Buffer  with  guards  is  shown  in 
Figure  3.8.  The  guard  attached  to  the  method  get,  (b. count  > 0),  indicates  that 
get  can  be  executed  only  when  the  buffer  is  not  empty. 

The  guard  attached  to  put,  (b. count  < size),  indicates  that  put  can  be  exe- 
cuted only  when  the  buffer  is  not  full.  The  complete  definition  of  Extended-Buffer, 
which  is  a subclass  of  Bounded-Buffer,  is  shown  in  Figure  3.9.  The  Extended-Buffer 
inherits  the  methods  including  the  guards  of  the  Bounded-Buffer.  The  inherited 
guards  work  coherently  with  the  new  guard  defined  in  pop.  □ 
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class  definition  Extended-BuMer(itesrtypa,  size) 
superclass:  Bounded JufferCiteatype,  size) 
inherit:  pat,  get,  is.empty,  length 
method  pop  b 

guard (b. count  > 0) 
expression 

C /?[front,dec]  b,  last (b. store)  ] 

end  class 

Figure  3.9.  The  complete  definition  of  the  class  Extended  .Buff  or. 

This  example  has  been  used  to  illustrate  the  difficulty  in  integrating  inheritance 
and  synchronization  constraints  in  some  object-oriented  parallel  models  [49,  78). 
The  above  example  demonstrates  inheritance  can  be  integrated  with  synchronization 
constraints  without  difficulty  in  PROOF.  By  separating  synchronization  constraints 
(guard)  from  the  behavior  of  methods  (expression),  integration  of  inheritance  and 
parallelism  can  be  achieved  with  interference.  Only  modification  of  the  bodies  is 
required. 

Since  each  method  is  associated  with  a guard  specifying  the  condition  under  which 
the  method  can  be  executed,  each  method  can  be  a natural  unit  of  inheritance. 

14 Persistence  of  Objects 

A major  deficiency  of  the  functional  paradigm  is  its  history-insensitivity.  PROOF 
is  made  history  sensitive  by  making  the  objects  persistent  and  allowing  the  reception 
of  values  by  the  objects,  i.  e.  the  assignment  of  values  to  the  objects.  The  local 
data  of  the  objects  is  persistent.  The  reception  of  values  by  the  objects  will  modify 
the  local  data  of  the  objects.  A pseudo-function  71,  called  the  reception  function , is 
introduced  to  denote  the  reception  of  a value  by  an  object. 


*M(e) 
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Object  Buffer:  instance  of  Bounded-Buffer  (my-item,  my-size) 

Active  Object  Producer : instance  of  Producer-Class  (my-item) 

Body  while  True  (72  [Buffer]  (put  Buffer  produce)) 

Active  Object  Consumer:  instance  of  Consumer-Class  (my-item) 

Body  while  True  ( ()[72  [Buffer],  consume)  (get  Buffer)) 

Figure  3.10.  The  definition  of  the  objects  for  producer-consumer  problem. 

72  is  not  a function,  but  can  be  treated  as  such.  72  has  two  parameters:  an  object  o, 
the  recipient,  and  the  expression  e,  to  be  received  by  o.  e may  contain  applications 
of  applicative  functions  only.  This  pseudo-function  can  only  appear  inside  bodies  of 
active  objects,  and  may  not  be  nested.  Major  differences  between  modification  of  the 
objects  through  72  and  traditional  assignments  are  : 

• The  evaluation  of  the  expression  e in  72  can  be  done  in  parallel,  since  e contains 
only  applications  of  purely  applicative  functions. 

• No  partial  modification  to  the  object  o is  allowed.  The  local  data  of  an  object 
can  only  be  modified  as  a whole  entity,  i.  c.  its  components  cannot  be  modified 
individually. 

objects  implies  that  all  the  methods  in  an  object  are  still  applicative,  i.  e.  there  are 
no  side-effects. 

This  restriction  effectively  preserves  the  referential  transparency  at  the  method 
Example  6.  Bodies  of  active  objects. 

The  complete  definition  of  the  objects  for  a producer-consumer  problem  is  shown  in 
Figure  3.10.  In  the  body  of  Producer,  put  adds  an  item  produced  by  the  method 
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produce  to  the  buffer,  and  returns  a new  buffer  as  its  result.  The  object  Buffer  will 
receive  the  value  of  the  new  buffer.  In  the  body  of  Consumer,  get  returns  two  values: 
a new  buffer  with  an  item  deleted  from  the  original  buffer  and  the  deleted  item.  The 
Buffer  will  receive  the  value  of  the  new  buffer  and  the  deleted  item  will  be  consumed 
by  the  Consumer  as  the  result  of  invoking  consume.  □ 

Simultaneous  access  to  the  objects 

An  object  can  participate  in  more  than  one  function  evaluation.  There  may 
be  several  attempts  to  modify  the  same  objects  at  the  same  time.  Simultaneous 
modification  of  the  objects  may  result  in  inconsistent  and  incorrect  states.  Thus, 
simultaneous  modification  to  the  same  object  must  be  serialised  so  that  at  any  given 
moment  an  object  will  be  a recipient  of  only  one  of  the  function  evaluations.  In  order 
to  illustrate  simultaneous  access  to  the  objects,  let’s  consider  the  methods  defined 
in  the  Bounded-Buffer  shown  in  Fig  3.8.  Simultaneous  invocations  of  length  and 
put,  or  length  and  get  are  permitted  since  only  one  of  the  methods  will  attempt 
to  modify  the  object.  On  the  other  hand,  simultaneous  invocation  of  put  and  get 
are  prohibited  since  both  the  methods  will  attempt  to  modify  the  object.  Thus,  they 
must  be  serialized. 

At  any  moment,  the  status  of  any  object  involved  in  an  expression  falls  into  one 
of  the  following  three  categories, 

rtad-only  : The  expression  only  needs  to  read  the  value  of  the  object. 
will-modify  : The  expression  will  modify  the  object,  but  the  modification  does  not 
occur  at  this  moment. 

modifying  : The  expression  is  currently  modifying  the  object. 

In  order  to  ensure  the  consistency  and  the  correctness  of  the  objects,  a multi-mode 
locking  mechanism  is  adopted.  There  arc  three  different  types  of  locks,  R-Lock, 


Table  3.1.  Multi-mode  locking  mechanism 


V-Lock  and  M-Lock,  that  are  associated  with  the  three  status  of  the  object,  read- 
only, will-modify  and  modifying,  respectively.  Before  evaluating  an  expression 
involving  an  object  o,  a proper  lock  tor  o must  be  obtained.  A lock  is  granted  only 
when  it  is  compatible  with  other  locks  granted  for  the  same  object,  according  to  the 
compatibility  chart  in  Table  3.1. 


The  functional  forms  used  to  express  parallelism  are  based  on  data  dependency. 
In  other  words,  the  application  of  functions  can  be  ordered  according  to  the  input 
and  output  relations.  However,  we  may  need  functions  whose  relations  are  not  di- 
rectly based  on  input-output  relations.  For  instance,  consider  a dining  philosophers 
problem.  In  this  problem  each  philosopher  either  thinks  or  eats  noodles.  In  order  for 


a philosopher  to  eat  noodles,  he  first  has  to  acquire  two  chopsticks.  A philosopher 
repeats  a sequence  of  activities:  think , acquire  chopsticks,  eat  noodles  and  release 
chopsticks.  If  we  consider  each  activity  to  be  a function,  each  can  be  designed  as  a 
method  in  PROOF.  In  this  case,  there  is  no  significant  data  flow  between  think  and 

acquire  chopsticks,  he  has  to  express  such  sequential  ordering  of  activities.  However, 
the  function  think  may  not  have  significant  output  generated  and  used  as  input  to 
the  function  acquire  chopsticks.  Thus,  if  we  do  not  have  a sequential  control  function, 


wc  need  to  create  data  flow  artificially  between  these  two  functions.  In  fact,  we  can 
use  a kind  of  flag  as  an  output  parameter  of  a preceding  function  to  indicate  the 
completion  of  the  preceding  function,  such  as  think.  The  same  flag  is  used  as  an 
input  parameter  to  trigger  the  following  function  acquire  chopsticks.  The  use  of  such 
flags  not  only  makes  programming  very  complicated,  but  also  reduces  readability  and 
modifiability  of  the  programs. 

To  alleviate  such  problems,  we  introduce  the  following  functions,  called  control 
functions.  Since  there  are  applications  which  require  explicit  control  structure  other 
than  data  flow,  we  need  additional  constructs  to  express  high  level  sequential  or  par- 
allel control  structures.  ; is  a sequential  control  function  defined  as  follows: 

a)  Control  function:  ; (called  sequential) 

«.| 


in  which  ; means  sequential  execution.  It  is  not  necessary  that  there  are  any  data 
dependency  relations  among  e,s. 

//  is  a parallel  constructs  defined  as  follows: 
b)  Control  function:  //  (called  parallel) 

//(« «.) 

= cll/e,ll...llc. 

in  which  //  means  parallel  execution.  These  control  functions  arc  used  to  specify 
sequential  control  flow  or  explicit  parallel  control  flow  among  functions  in  the  bodies 
of  the  objects. 
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In  addition  to  these  explicit  control  functions,  we  also  define  while  and  if  control 
functions  as  follows: 

c)  Control  function:  if  (called  conditional  select) 

if  \p  f jlM 

f /(*)  ifp(*)istrue 
| g(x)  if  p(x)  is  false 

in  which  p,  / and  j arc  of  type  T boolean,  T -»  Tj,  and  T -*  T„  respectively,  and 
i is  of  type  T. 

d)  Control  function:  while  (called  conditional  loop) 
while  [p  J)\x) 

_/  while\p  f\{f(x)]  if  P(x)  is  true 
“ \ x if  p(x)  is  false 

in  which  p and  / are  of  type  T — boolean,  T -»  T,  respectively,  and  x is  of  type 
T.  The  control  functions,  if  and  while  are  applicative  functions  without  side-effects. 
They  can  be  used  in  the  method  definition  as  well  as  in  the  body. 

3.6  Parallelism  in  PROOF 

In  this  section,  we  discuss  the  parallelism  offered  by  PROOF  and  its  semantics. 
3.6.1  Sources  of  Parallelism 

PROOF  offers  parallelism  primarily  in  two  different  levels:  the  object  level  and 
the  method  level.  The  object  level  parallelism  is  achieved  by  allowing  more  than  one 
object  to  be  active  at  a time,  and  the  method  level  parallelism  can  be  achieved  by 
defining  methods  as  applicative  functions. 

Each  of  the  active  objects  can  invoke  a method,  thus  there  can  be  a number 
of  methods  being  executed  simultaneously.  The  potential  for  the  simultaneous  par- 
ticipation of  an  object  in  the  evaluation  of  different  functions  also  implies  another 


source  of  parallelism  in  PROOF.  Method  level  parallelism  is  obtained  from  the  follow- 
ing two  sources:  parallel  evaluation  of  arguments  of  functions  and  parallel  evaluation 
of  functions  or  replication  of  a function.  The  former  is  achieved  because  of  referential 
transparency  of  applicative  functions.  The  latter  is  achieved  by  functional  forms, 
such  as  a and  0.  In  the  following,  we  summarize  the  sources  of  parallelism  in  the 
computation  model.  Because  all  the  functions  in  PROOF  are  applicative,  parallelism 
is  obtained  from  the  following  two  sources: 

PI  Parallel  evaluation  of  arguments  of  functions:  It  is  made  possible  because  of  the 
referential  transparency  of  applicative  functions. 

P2  Parallel  evaluation  of  a number  of  functions  or  replications  of  a function:  It  is 
made  possible  by  functional  forms  such  as  a and  0. 

The  active  objects  introduce  another  source  of  parallelism  in  PROOF: 

P3  There  can  be  a number  of  the  objects  that  are  active  simultaneously  throughout 
the  execution. 


P4  An  object  can  be  involved  in  two  or  more  function  evaluations  simultaneously. 


In  this  section,  we  will  define  the  semantics  of  PROOF  in  terms  of  its  parallel 
behavior.  We  give  the  semantics  to  PROOF  at  three  different  levels:  one  is  at  the 
method  definition  level  (applicative  function),  another  is  at  the  object  definition  level 
and  the  other  is  the  interface  level  between  the  two  levels. 


are  related  to  the  issue  of  the  semantics  of  functional  programming  languages.  At  the 
method  definition  level,  an  important  concern  is  how  to  select  the  expression  to  be 
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executed  next.  There  are  three  different  semantics  for  programs  written  with  func- 
tional programming  languages,  according  to  how  an  expression  to  be  executed  next  is 
chosen:  strict,  lazy  and  lenient.  Strict  semantics  imposes  the  following  requirement 
during  evaluation  of  the  expression:  complete  evaluation  of  all  arguments  before  a 
function  call  and  complete  evaluation  of  all  components  before  a structure  is  built. 
This  requirement  implies  that  strict  semantics  may  require  useless  computation  that 
is  not  necessarily  to  be  executed.  This  has  two  disadvantages:  (1)  the  consumption  of 
resources  and  (2)  a failure  to  produce  a solution.  It  also  requires  additional  effort  to 
implement  in  distributed  memory  parallel  computers,  since  some  of  the  unnecessary 
results  may  complicate  their  handling  and  increase  overhead  in  communication  costs. 

In  contrast  to  such  strict  semantics,  nonslrict  semantics  does  not  impose  any 
requirement  mentioned  above.  Both  la2y  and  lenient  semantics  arc  nonstrict  in  the 
sense  that  any  of  the  above  requirements  for  strict  semantics  is  not  required.  Instead, 
lazy  semantics  imposes  the  following  requirement:  a computation  cannot  be  executed 
unless  it  contributes  to  the  final  answer.  This  ensures  that  function  arguments, 
structure  components  and  arms  of  conditionals  are  not  evaluated  unless  they  are 
needed.  Furthermore,  termination  of  the  computation  and  obtaining  the  final  answer 
occurs  at  the  same  time,  since  there  will  be  no  further  reductions  required  when 
a normal  form  is  reached.  As  a consequence,  unlike  the  case  of  strict  semantics, 
lazy  semantics  guarantees  minimal  computation  to  obtain  a final  answer  if  it  exists. 
However,  the  lazy  semantics  may  require  a substantial  amount  of  work  to  determine 
the  next  computation  to  be  executed. 

Lenient  semantics  (79)  imposes  the  following  requirement:  complete  evaluation 
of  the  predicate  before  any  arm  of  a conditional  is  considered  for  execution.  Thus, 
unnecessary  computations  are  avoided  only  when  they  are  guarded  by  appropriate 
conditions.  For  instance,  consider  the  control  function  if\pf  p]  [zj.  Under  lenient 
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semantics,  first  p(x)  will  be  evaluated.  Then,  based  on  the  result  of  the  evaluation, 
either  f(x)  or  g(z)  will  be  chosen  to  execute.  When  the  expression  is  executed  based 
on  strict  semantics,  the  three  subexpressions,  p(x),  /( x)  and  g(x),  can  be  executed  in 
parallel,  since  lenient  semantics  can  be  considered  as  a compromise  between  strict 
semantics  and  lazy  semantics.  Lenient  semantics  can  eliminate  substantial  effort  in 
determining  the  next  computation  to  execute.  It  can  also  initiate  many  computations, 
and  thus  increase  the  opportunities  for  parallel  execution.  In  PROOF  we  adopt  the 
lenient  semantics  due  to  the  advantages  for  parallel  processing.  In  the  following, 
we  give  the  semantics  of  PROOF  in  its  parallel  aspects.  The  key  element  in  the 
definition  of  the  semantics  is  the  pamllelieer  denoted  Para,  which  describes  how  a 
program  in  PROOF  can  be  executed  in  parallel.  The  parallelizer  Para  is  defined 
using  four  evaluation  schemata  presented  in  the  following  order: 

Schema  A:  evaluation  of  applicative  functions 
Schema  B:  evaluation  of  the  pseudo-function  R 
Schema  C:  evaluation  of  control  functions 

Schema  D:  evaluation  of  a set  of  the  objects  which  composes  a program  in 


It  is  shown  that  the  parallel  execution  of  a program  according  to  the  parallelizer 
Para  is  equivalent  to  a sequential  execution  of  the  same  program. 

Schema  A : Let  e be  an  expression  consisting  of  applications  of  applicative 
functions  only.  Para(e)  is  defined  as  follows. 

Case  1:  e is  a constant 

Case  2:  e is  a function  /(ei,C2,...,e„)  except  the  functions  if  and  while 
Para(/(e1,e„...,e.))  = 


parbegin 


u,  <-  Para  (ea)||v2  •-  Para  (ea)|| . . . ||v„  <-  Para  (e„) 

parend 
0 *-/(»!, 

Case  3:  e is  a function  if  \p  f s|[x] 

Para(i/[  p f g j[xj)  = 
if  Par.fpfx))  then 

u - Para(  f(x)  ) 

v - Para(  g(x)  ) 

Case  4:  e is  a function  while  [p  /I(x) 

Parafu <hilc  [ p f )[x])  = 
if  Para(p(x))  then 

while  (p  f )|/(x)| 


Here,  parbegin  . . . . parend  is  ust 

of  elements  separated  by  ||.  The  Case  3 shows  that  PROOF  supports  the  lenient 
semantics  by  selecting  only  one  arm  for  execution  after  evaluating  a conditional.  If  we 
treat  the  if  control  function  as  a general  function  and  apply  it  to  the  semantics  in  the 
case  2,  the  strict  semantics  can  be  obtained.  We  assume  that  Parafr,)  terminates1. 
However,  in  real  execution  e,  may  not  terminate.  In  such  a case,  we  can  use  a kind  of 
fair  scheduling  strategy.  That  is,  we  can  prevent  such  non*terminating  process  from 
'We  ten  easily  extend  this  to  non-slricl  semantics  by  evaluating  / in  parallel  with  Parafr,). 
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exhausting  the  system  resources  by  limiting  service  time  for  such  process.  With  this 
provision,  lenient  semantics  can  produce  the  same  answers  as  lazy  semantics  with 
only  a bounded  amount  of  excess  computation. 

Variations  of  schema  A can  be  derived  for  the  functional  forms  a,  0,  7,  i and  >; 
according  to  their  definitions. 

Schema  A„  : Let  e be  an  expression  a/[xi,xa,...,x»l 

Para(a/[xi,  xj x.])  = 

parbegin 

V|  — Para(/(xi))||i>i «-  Para(/(xj))|| 

...K-Paxa(/(x„)) 

return  [vi, vj, . . . , v„] 

Schema  As  : Let  c be  an  expression  /3[/i,/a,. . .,/.|[xi,xj, • • • ,x„) 

Para(/J|/„/, /.]  |*.,*, x„])  = 

parbegin 

t>i  — Para(/i(xi))||i>j  — Para(/j(xa))|| 

...IK- Para (/„(*„)) 

parend 

return  [»i,»a »»] 

Schema  A,  is  omitted  since  it  can  be  considered  as  a special  form  of  Ag. 

Schema  Aj  : Let  e bean  expression  d[/i,/j,. . • ./„]|x] 

Par»(«|/„/„..., /.](*])  = 
parbegin 

u,-Par^/1(x)K-P«»(/s(x)ll 


...||un«-l>ar»(/n(i)) 


parend 

return  [o,,uj v„| 

Schema  A„  : Let  e be  an  expression  i)|/](xi,  Xj, . . . , r„) 

Para(ij[/][xi,  x.])  s 

« Para(/(xi,  Para(/(xj Para(/(x„. „ Para(/(x„-, , Para(/(x. ) ) . . .) 


In  the  following,  we  prove  the  schemata  to  be  correct.  The  correctness  of  schemata 
A,  A„,  Ap,  A,,  As  and  A,  is  justified  in  the  following  theorem  and  lemma. 

Theorem  S.6.1  The  evaluation  o/ applicative  functions  in  parallel  described  in  schema 
A is  equivalent  to  a sequential  evaluation  of  the  same  functions. 

Proof  of  Theorem  3.6.1  Since  e consists  of  applications  of  applicative  functions  only, 
there  is  no  side-effect.  The  evaluation  order  of  the  componenls(subcxpressions)  of 
a function  does  not  affect  the  result  of  the  function  (Churvh-Rosser  theorem  [18]). 
Therefore,  it  implies  that  the  parallel  evaluation  of  all  the  subexpressions  will  yield 
the  same  result  as  the  sequential  evaluation  of  the  subexpressions  in  any  order. 

This  theorem  shows  that  the  evaluation  of  applicative  functions  is  equivalent  to  a 
sequential  evaluation  of  the  same  program. 

lemma  S.6.1  The  evaluation  of  applicative  functions  in  parallel  described  in  schemata 
Aft  , Ag,  Aft,  As  and  A„  is  equivalent  to  a sequential  evaluation  of  the  same  functions. 


Proof  of  lemma  3.6. 1 Since  all  the  functional  forms  belong  to  the  applicative  func- 
tions, by  Theorem  3.6.1  it  is  proved. 


PROOF  exploits  coarse  grain  parallelism  by  allowing  more  than  one  object  to  be 
active  at  a time.  Unlike  the  evaluation  of  applicative  functions,  parallel  execution  of 
the  objects  involves  parallel  modifications,  which  must  be  controlled  in  such  a manner 
that  only  the  results  equivalent  to  a series  of  sequential  modifications  of  the  objects 
are  permissible.  The  locking  mechanism  used  in  the  concurrency  control  of  data  base 
systems  is  adopted  here  to  ensure  the  scrializability  (63)  of  parallel  modifications  of 
the  objects.  R-Lock  is  used  to  indicate  that  an  expression  only  needs  to  read  an 
object  (read-only).  W-Lock  is  used  to  indicate  that  an  expression  will  modify  an 
object,  but  not  now  (will-modify).  H-Lock  indicates  that  an  expression  is  currently 
modifying  an  object  (modifying).  The  compatibility  between  these  locking  modes 
are  shown  in  Table  3.1. 

The  modification  of  the  objects  is  achieved  by  the  pseudo-function  7?.  The  execu- 
tion of  72 1 o 1(e)  is  described  in  schema  B.  First,  the  following  notation  is  introduced. 
Lets 


obj(e)=  e if  = i» 

l Ur„,obj(e,)  if  e = 
Schema  B : Let  o be  an  object  na 
defined  as  follows. 

Case  1:  e is  a constant. 

Para  (72  ( o ](«))  s 
Bl.l  : H-Lock({o}) 

B1.2:  o •—  e 


in  expression.  Para  (72  [ o ](e))  is 


Para  (72  [o  1(e))  = 
B2.1  : repeat 
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B2.2 : W-Lock({o}) 

B2.3  : R-Lock(obj(e)  - {o}) 

B2.4  : evaluate  guards  associated  with  the  methods  io  e 

B2.5  : if  one  or  more  guards  arc  False 

B2.6:  Unloek(obj(e)U{o}) 

B2.7  : until  all  guards  are  True 
B2.8 : t >-  Para(e) 

B2.9  : M-Lock({o)) 

B2.10:  Unlock(obj(e)-  {o}) 

B2.12:  Unlock({o}) 

In  case  1,  we  only  need  to  request  an  H-Lock  for  the  object  o before  the  modification, 
and  unlock  it  after  the  modification.  In  case  2,  an  R-Lock  is  requested  for  each  object 
involved  in  the  expression  e except  the  object  o,  for  which  a V-Lock  is  requested.  Then 
all  the  guards  associated  with  the  methods  in  e are  evaluated.  If  not  all  of  the  guards 
evaluate  to  True,  we  unlock  all  the  objects  and  repeat  this  evaluation  until  all  the 
guards  are  True.  Then,  after  evaluating  the  expression  e,  we  request  an  K-Lock  for 
an  object  o and  modify  the  object  o.  Finally  we  unlock  the  object  o. 

Theorem  .1.6.2  Schema  B ensures  that  the  parallel  modification  of  the  objects  is  equiv- 
alent to  a sequential  modification  of  the  objects,  i.  e.  the  parallel  modification  of  the 
objects  is  serialisable. 

Proof  ol  Theorem  S.6.S  In  [S6],  a sufficient  condition  for  serialisability  is  given  - the 
two-phase  locking  protocol.  The  two-phase  locking  protocol  requires  that  the  locking 
process  consists  of  two  phases,  growing  and  shrinking  phases.  In  the  growing  phase, 
locks  are  obtained,  but  cannot  be  released  until  we  complete  this  phase.  In  shrinking 


phase,  locks  an  nleaseJ,  and  no  locks  can  be  obtained,  including  upgrading  of  locks. 
In  other  words,  no  lock  can  be  obtained  after  we  start  to  nlease  locks.  Thenfon  we 
only  need  to  show  that  schema  B satisfies  the  two-phase  locking  protocol. 

Case  1:  When  e is  a constant,  only  one  lock  is  involved.  Thus,  it  is  obvious  that  it 
satisfies  the  two-phase  locking  protocol. 

Case  2:  When  e is  an  expression  in  which  one  or  more  objects  an  involved,  the 
growing  phase  consists  of  B2.S  to  B2.9,  and  the  shrinking  phase  consists  of 
Be,  10  to  BB.ie.  In  fact,  the  repeat  statement  in  the  growing  phase  may  regain 
unlocking  all  of  the  objects.  However,  the  two-phase  locking  protocol  is  not 
violated,  since  once  the  objects  an  unlocked,  we  will  not  enter  the  modifying 
ngion  (BS.IO  to  BB.ie).  When  we  eventually  enter  the  modifying  region,  we 
will  not  request  any  lock  after  we  start  to  nlease  locks.  Thus,  it  satisfies  the 
two-phase  locking  protocol.  Thenfon,  the  parallel  modification  of  the  objects  in 
PROOF  is  equivalent  to  the  sequential  modification  of  the  objects. 

We  give  the  semantics  to  the  explicit  control  functions  in  the  following. 

Schema  C, : Let  e be  an  expression  ;(e,,e2,...,e„) 

Para(;(e„ea c„))  = 

seqbegin 

return  ui, 
v,  - Para(ej), 
return  vj, 


Here,  seqbegin seqend  is  used  to  indicate  the  sequential  execution  of  elements 
separated  by  a comma. 

Schema  C// : Let  e be  an  expression  //(ej,ej, 

Para(//(ei,ej e»))  = 

parbegin 

seqbegin 

return  u, 
seqend 
seqbegin 

return  vj 
seqend 

seqbegin 

o„  Para(e„), 


Note  the  difference  between  Schema  Ap  and  Schema  C//.  In  the  latter,  it  is  not 
necessary  to  wait  for  all  the  results.  However,  in  the  former,  the  semantics  requires 
collection  of  the  results  before  proceeding  its  computation. 


Theorem  S.6.S  The  evaluation  of  the  explicit  control  functions  described  in  schemata 
C;  and  C//  is  equivalent  to  a sequential  evaluation  of  the  same  functions. 

Proof  ot  Theorem  .1.6.3  In  the  case  of  the  sequential  control  function  ■„  the  proof  is 
trivial.  In  the  case  of  the  parallel  control  function  //,  let  a body  of  an  object  o, 
body(3)  be  on  expression  //(ei.e, e„).  Wien  the  body(S)  includes  only  applica- 

tive functions,  by  Theorem  S.6.1  the  parallel  evaluation  is  equivalent  to  a sequential 
evaluation.  When  the  body(o)  includes  the  pseudo-function  or  the  pseudo-functions, 
the  parallel  evaluation  is  equivalent  to  a sequential  evaluation.  Therefore,  the  eval- 
uation of  the  control  functions  i and  //  in  parallel  is  the  equivalent  of  a sequential 
evaluation  of  those  functions . 

Note  that  Theorem  3.6.3  does  not  guarantee  that  the  parallel  evaluation  of  the 
body  always  gives  the  same  result.  Theorem  3.6.3  ensures  that  it  is  always  possible  to 
find  a sequential  evaluation  which  is  equivalent  to  a parallel  evaluation  of  the  body. 
For  instance,  when  the  body  involves  more  than  one  modification  statement  to  the 
same  object,  i.e.  pseudo-function  R whose  recipient  is  the  same  object,  the  result 
varies  depending  on  the  execution  order  of  the  pseudo-function  statements.  In  order 
to  ensure  that  the  parallel  evaluation  of  the  body  always  yields  to  the  same  result, 
we  can  make  a restriction  on  the  definition  of  the  body. 

Definition  3.6.1  4 body  is  called  safe  if  and  only  if  there  is  at  most  one  modification 
statement  for  each  object  involved  in  the  body. 

If  a body  of  an  object  is  safe,  all  the  expressions  in  that  body  can  be  executed  in 
parallel.  Thus,  parallel  evaluation  guarantees  the  same  result  all  the  time. 

Lemma  $ 6.2  The  parallel  evaluation  of  a safe  body  * 
equivalent  to  a sequential  evaluation  of  that  body. 


unique  result  which  is 


Schema  D describes  the  execution  of  a set  of  the  objects,  that  compose  a program 
in  PROOF. 

Schema  D:  Let  O be  a set  of  the  objects  which  compose  a program,  and  O = 
be  a subset  of  the  objects  that  are  non-passive.  We  use  bodxfidi)  to 
denote  the  body  of  a non-passive  object  5,.  Note  that  bodyloi)  is  a function  which 
may  contain  the  pseudo-function  H.  The  parallel  execution  of  O is  defined  as  follows: 

Para(O)  = 
parbegin 

Para(6odjr(oi))||  Para(Wy(oj))|| . . . ||  Para(6ody(o„)) 
parend 

Theorem  S.6.1  Schema  D ensures  that  the  parallel  execution  of  a propram  in  PROOF 
is  equivalent  to  the  sequential  execution  of  the  program. 

Proof  of  Theorem  S.6.1  According  to  Theorem  S.6.1,  any  evaluation  of  the  applica- 
tive functions  in  parallel  is  equivalent  to  the  sequential  evaluation  of  the  same  func- 
tions. According  to  Theorem  S.6.1,  any  evaluation  of  the  pseudo-function  U in 
parallel  is  serializable.  According  to  Theorem  S.6.S,  any  evaluation  of  a body  is 
equivalent  to  a sequential  evaluation  of  the  body.  Therefore,  the  parallel  modification 
involved  in  the  set  of  the  bodies  is  also  serializable. 


INTERMEDIATE  PROGRAM  REPRESENTATION  FOR  PROOF/L 

There  arc  several  intermediate  languages  for  functional  language  implementation, 
such  as  IF1  [73],  P-TAC  |6],  and  Lean  (10).  IF1,  which  was  developed  for  the  imple- 
mentation of  the  functional  language  SISAL,  is  a hierarchical  graph  language  that 
describes  the  dataflow  graphs  produced  from  SISAL  functions.  P-TAC  is  an  interme- 
diate language  designed  to  capture  the  sharing  of  computation  so  that  optimisation  of 
ID  programs  can  be  unambiguously  understood  and  classified  during  the  compilation 
of  ID  programs.  Lean,  which  is  an  intermediate  language  for  specifying  computation 
in  terms  of  graph  rewriting,  is  not  aimed  at  any  specific  functional  language,  and 
it  can  serve  as  an  intermediate  form  between  functional  and  machine  languages.  In 
general,  these  intermediate  languages  serve  as  a basis  for  code  optimisation  analysis. 

The  Intermediate  Program  Represcnlation(IPR)  language  is  designed  to  represent 
the  parallelism  in  the  PROOF/L  program  and  analyze  it  for  the  efficient  exploitation 
on  various  parallel  computers.  The  programming  language  PROOF/L  is  a parallel 
object-oriented  functional  language  based  on  the  computation  model  PROOF.  Once 
a program  is  written  in  PROOF/L,  the  program  is  transformed  to  a target  code  via 
a two-level  translation.  To  generate  a target  code  for  a specific  parallel  processing 
system,  a PROOF/L  code  is  first  translated  into  an  IPR,  then  the  IPR  is  translated 
to  a target  code.  The  front-end  translation  makes  all  the  parallelism  in  the  program 
explicit.  Because  this  step  is  machine-independent,  it  needs  to  be  done  only  once 
for  each  PROOF/L  program.  The  back-end  translation  is  machine  dependent  and 
the  analysis  to  obtain  efficient  target  codes  can  be  incorporated  at  this  stage.  This 


two-level  transformation  approach  is  shown  in  Figure  4.1.  This  chapter  is  organized 
as  follows:  In  Section  4.1,  the  general  characteristics  of  the  IPR  is  presented.  In  Sec- 
tion 4.2,  the  formal  definition  of  the  IPR  is  given  using  the  Petri-net.  In  Section  4.3, 
the  transformation  rules  from  the  PROOF/L  program  to  the  IPR  and  from  IPR  to 
the  target  program  are  presented. 
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PROOF/Lcode 


4.1  Characteristics  of  the  IP£ 


The  IPR  consists  ot  two  different  types  of  representation;  one  is  a Petri-net  and 
the  other  is  a set  of  function  nodes  and  their  relations  which  we  will  introduce  in  this 
section.  The  semantics  for  the  IPR  is  also  given  in  the  two  different  levels;  object  level 
and  method  level.  The  object  level  semantics  gives  meaning  to  the  object  bodies  of 
the  PROOF/L  program.  The  method  level  semantics  gives  meaning  to  the  function 
nodes  used  to  represent  the  methods  in  the  PROOF/L  program.  This  two  level 
semantics  makes  it  easy  to  understand  the  important  issues  in  parallel  programs, 
such  as  communication/synchronization  aspects  without  considering  the  unnecessary 
details  of  the  program.  This  separation  of  the  semantics  also  allows  the  verification 
of  programs  in  different  levels,  and  thus  the  complexity  of  the  understanding  and  the 
verification  of  programs  can  be  significantly  reduced.  Before  we  present  the  formal 
definition  of  the  IPR,  we  summarize  the  characteristics  of  the  IPR  as  follows: 

• Graphical  representation 

A graphical  representation  can  help  conceptual  understanding  of  parallel  pro- 
grams on  a construct-by-construct  basis.  A PROOF/L  program  is  represented 
as  a hybrid  graph  on  the  two  different  levels:  object-level  and  method  level. 
At  the  object-level,  the  IPR  is  a Petri-net,  which  is  a bipartite  directed  graph. 
It  is  used  to  represent  the  method  invocation  structure  of  the  program.  At 
the  method  level,  the  IPR  is  a directed  graph  in  which  each  node  represents  a 
computation  and  each  directed  arc  represents  precedence  relationships  between 
two  nodes.  The  nodes  can  be  divided  into  two  types  - computation  node  and 
non-computation  node.  The  compulation  node  represents  a function  receiving 
input  value(s)  and  generating  output  value(s).  These  functions  arc  side-effect 
free,  that  is,  they  always  produce  the  same  result  when  the  same  input  values 


»re  given.  The  non-computation  node  does  not  represent  a specific  function, 
but  they  Me  required  to  exist  to  represent  computation  in  the  IPR.  A set  of 
edges  between  nodes  specifies  the  data  dependency  between  functions. 

• Data  and  control  dependency  representation 
The  precedence  relationships  among  computations  at  the  method-level  are 
based  on  the  data  dependency  among  them.  A computation  associated  with  a 
node  can  be  executed  when  the  input  data  associated  with  incoming  mc(s)  is 
available,  and  after  its  execution  the  result  is  available  for  its  successor  node(s). 
The  functional  style  of  PROOF/L  permits  to  extract  data  dependency  relation- 
ships easily  among  the  computations.  The  dependency  relationships  among 
objects  are  based  on  control  dependency  in  the  sense  that  the  method  is  not 
invoked  by  the  availability  of  data  but  invoked  by  the  necessity  of  that  method 
execution.  Thus,  we  can  also  regard  this  control-dependency  among  objects  as 
a demand-driven  approach.  On  the  other  hand,  the  data  dependency  among  the 
computations  in  the  method-level  can  be  regMded  as  a data-driven  approach. 
The  effect  of  such  separation  of  dependency  relations  needs  to  be  studied  more. 

• Partitioning  analysis 

The  IPR  can  be  used  to  analyse  pMlitioning  and  allocation  of  processes  to 
processors.  An  important  factor  to  minimize  the  execution  time  of  a program 
is  to  determine  the  proper  size  of  the  tasks  to  be  assigned  as  a sequential  code  to 
physical  processors.  To  determine  the  proper  grain  size,  we  need  to  analyze  the 
tradeoff  between  communication  overhead  occurring  due  to  pMallel  execution 
of  processes.  In  order  to  use  the  IPR  for  this  analysis,  we  need  to  annotate  the 
graph  with  information  such  as  execution  time  and  communication  time.  The 
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use  of  the  1PR  for  grain  size  analysis  and  clustering  will  be  explained  in  the 
next  chapter. 

it  IPR  Definition 

The  IPR  is  a hybrid  graphical  program  representation  in  which  the  high-level  con- 
trol structure  is  specified  as  a Petri-net  and  in  which  the  functionality  is  specified  as 
. data  dependency  graph.  The  high-level  control  structure  corresponds  to  the  bodies 
of  the  objects  and  the  low-level  functionality  corresponds  to  the  method  definitions 
within  each  object.  The  high-level  structure  is  called  the  object-level  IPR  and  the 
low-level  representation  of  functionality  is  called  the  method-level  IPR. 
l?l  Object-level  IPR 

The  IPR  in  the  object-level  is  represented  as  a Petri-net  [65).  A Petri-net  repre- 
sents the  static  structure  of  the  system  and  its  dynamic  behavior.  The  static  structure 
is  represented  as  a net  structure  N = (P,T,  F),  in  which  P is  a set  of  places,  repre- 
senting the  availability  of  data  or  control,  T is  a set  of  transitions,  representing  the 
activities,  and  F is  a flow  relation,  representing  the  dependencies  between  the  places 
and  the  transitions.  To  specify  the  dynamic  behavior  of  the  system,  the  state  of  the 
system  is  represented  by  a marking  in  a Petri-net,  which  is  a distribution  of  tokens 
over  places  of  the  net. 

We  define  the  Petri-net  to  be  used  as  the  object-level  IPR  for  the  PROOF/L 
programs  in  the  following: 

n.Snilian  l.S.l  The  Petri-net  Tti  is  defined  as  a 5-tuple  such  that 
T>M  = (P,T,F,I,D) 


P is  a set  of  places, 


T is  a set  of  transitions, 

F C (P  x T)  U (T  X P)  is  a set  of  arcs  (flow  relation), 
/ is  a set  of  initial  places,  and 
D is  a set  of  final  places. 


Definition  J.2.2  The  preset  of  a transition  t is  a set  of  places  such  that 
at  = {p|p  £ P such  that  (p,t)  e F). 

Definition  J.S.3  The  postsct  of  a transition  l is  a set  of  places  such  that 
f = {p|p  € P such  that  (l,p)  € F). 

Definition  J.S.i  The  preset  of  a place  p is  a set  of  transitions  such  that 
•p={t|ier  such  that  (l,p)  € F). 

Definition  t.2.5  The  postsct  of  a place  p is  a set  of  transitions  such  that 
pa  = {t|f  6 T such  that  (p.t)  € F). 

In  Pfif,  there  is  at  least  one  place  p such  that  op  = a.  There  also  is  at  least  one 

place  p such  that  pa  = o.  Let  num  be  a function  such  that  num(p)  is  number  of 

Definition  I.S.6  The  initial  place  is  a place  p such  that  peP  and  num(p)  > 0 and 

Definition  J.S.1  The  final  place  is  a place  p such  that  p € P and  pa  = e. 

A marking  in  the  Petri-net  is  changed  according  to  the  following  transition  rules: 

1)  A transition  t is  said  to  be  enabled  if  each  input  place  in  •<  is  marked. 

2)  A firing  of  the  enabled  transition  transition  t removes  a token  from  each  input 


place  p in  at,  and  adds  a token  to  each  output  place  it 


When  transitions  represent  computations  that  are  too  complex  or  places  that  re- 
quire significant  transformation,  the  transitions  or  the  places  can  be  further  refined. 
Let  the  refinement  of  a transition  be  a sub-net  sn.  The  transition  can  be  refined 
according  to  the  following  rules: 

a)  The  incoming  links  and  the  outgoing  links  of  the  transition  serve  as  input  and 
output  parameters,  respectively. 

b)  There  is  only  one  transition  that  receives  all  the  input  parameters. 

c)  There  is  only  one  transition  that  produces  all  the  output  parameters. 

d)  All  the  transitions  except  these  two  transitions  and  places  can  interact  only  with 
the  places  and  the  transitions  defined  within  the  sub-net  sn. 

The  place  can  be  refined  according  to  the  same  rules  as  these  except  that  the  tran- 
sition is  replaced  with  the  place. 

There  are  two  approaches  for  graphically  representing  programs:  a task  interac- 
tion graph  and  a task  precedence  graph.  In  the  task  interaction  graph,  a program  is 
represented  as  a undirected  graph  in  which  each  node  represents  a task  and  each  edge 


directional,  the  execution  order  relation  among  the  tasks  cannot  be  predicted.  The 
known  or  estimated  communication  time  can  be  specified  by  lumping  together  all  of 
the  communication  time  required  during  the  program  execution  between  processors. 

The  task  interaction  graph  is  suitable  for  an  analysis  of  task  allocation  whose  goal 
is  to  minimize  the  total  execution  time  and  communication  time.  In  the  task  prece- 
dence graph,  a program  is  represented  as  a collection  of  tasks  and  explicit  execution 
dependencies  expressed  in  the  form  of  precedence  relationships.  This  representation 
is  useful  for  modeling  a program  whose  behavior  can  be  statically  known  in  advance. 


Figure  4.2.  Precedence  relations  among  functions 
In  both  graphical  representations,  the  concept  of  the  shared  data  does  not  exist,  and 
even  simple  synchronization  cannot  be  specified.  For  instance,  suppose  that  we  have 
three  functions,  ft,  fS  and  sharcd-func , whose  relations  are  shown  in  Figure  4.2.  One 
interpretation  for  the  relations  shown  in  Figure  4.2  is  that  the  function  sharcd-func 
requires  both  the  results  of  ft  and  fS  as  input  in  order  to  execute  sharcd-func.  We 
call  this  dependency  relation  as  value  flow.  However,  there  may  be  situations  such 
that  sharcd-func  may  need  only  one  input  from  either  of  the  two  functions.  Suppose 
that  //  and  fS  arc  clients  requesting  a service  to  sharcd-func.  If  sharcd-func  can  serve 
only  one  client  at  a lime,  the  task  precedence  graph  cannot  represent  this  kind  of 
relations  among  tasks.  We  call  this  relation  as  thread  flow. 

In  the  task  interaction  graph,  such  relationship  could  be  implicitly  represented. 
However,  due  to  the  lack  of  a precedence  relationship,  analysis  using  precedence 
relationships  cannot  be  performed  on  that  graph.  On  the  other  hand,  the  Petri-net 
can  easily  distinguish  the  differences  among  those  cases,  as  shown  in  Figure  4.3.  The 
Petri-net  is  chosen  due  to  its  capability  to  distinguish  the  thread  flow  and  the  value 
flow.  The  problems  with  the  Petri-net  is  that  recursive  behavior  cannot  be  given  the 
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Figure  4.3.  The  Petri-net  representation  of  (a)  value  flow  (b)  thread  flow 
exact  semantics.  Addition  of  the  inhibitor  arc  to  the  Petri-net,  however,  can  increase 
the  express  power  so  that  the  semantics  of  the  recursive  behavior  can  be  exactly 
represented. 

4.2.2  Method-level  IPR 

The  method-level  IPR  is  a data  dependency  graph.  In  the  object-level  IPR  each 
method  is  represented  as  a transition.  In  the  method-level  IPR  functionality  corre- 
sponding to  each  transition  in  the  object-level  IPR  is  represented  as  a set  of  nodes 
and  their  dependency  relations.  We  present  function  nodes  of  the  method-level  IPR 
and  give  the  semantics  using  the  Petri-net.  We  define  the  method-level  IPR  in  the 
following  manner: 

Definition  i.2.8  The  method-level  IPR  is  a directed  graph  Q such  that 
S = (V,E) 
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Vi  gl'ua  function  node,  and 

(vi,Vj)  € E is  an  edge  representing  data  dependency  from  V;  to  v,. 
The  method-level  IPR  is  composed  of  the  following  nodes: 

• A simple  function  node  vj  6 V represents  a primitivcfunction  such  as  *, -,  + .... 
The  input  and  output  for  v/  can  be  represented  as  follows: 

»l : A.  /ji  • - • > /■  -•  0 in  which  m > 1. 

• A constant  node  vei  € V represents  a constant  value  generator,  which  produces 
the  specified  same  valuc(s)  all  the  time.  There  is  no  input  to  this  v„  type  of 

Vc»  0 which  0 is  a constant  value. 

• An  id  node  v,  £ V represents  an  identity  function,  which  always  returns  input 
as  output. 

v, : / 0 in  which  1 = 0. 

• A copy  node  v^  € V represents  a duplicator,  which  receives  an  input  and 
produces  the  appropriate  number  of  copies  having  the  same  value  as  that  input. 

Vcp  : / — » Oi, . . . , Om  in  which  value  of  / = value  of  Oi  for  1 < i < m. 

• A macro  function  node  vm/  € V represents  a compound  function  composed  of 
simple  and/or  macro  functions. 

“n>/ : /i, fj, 0,  for  m > 1. 

receives  input  data  /i,  /*, . . . , 1m  and  control  data  e and  returns  an  input  as 
an  output  according  to  the  value  of  control  data  c. 
v.:/„/, lm,e->0. 
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• A distributor  node  vj  € V represents  a conditional  construction  function,  vs 
receives  input  data  / and  control  data  c and  passes  / to  one  of  output  port  O, 
according  to  the  value  of  c. 

ej : /,  e -t  (0|,  Os, , 0„)  in  which  (...)  means  one  of  the  items  within 
(. . .)  is  chosen. 

• A merge  node  um  € V represents  a nondeterministic  selector,  which  receives 
an  arbitrary  number  of  input  data  at  a time  and  returns  one,  which  arrives 
first.  If  more  than  one  input  arrives  at  the  same  lime,  a single  input  is  chosen 
arbitrarily. 

-0. 

t A split  node  v,  € V represents  a decomposition  function,  which  decomposes 
input  data  and  returns  a set  of  data  decomposed. 

V,:  / -»  01, 0],..., On.. 

• A construct  node  vc  6 V represents  a composition  function,  which  composes  a 
set  of  input  data  and  returns  them  as  one  clement. 

ve:/l,/j U-0. 

The  selector  and  distributor  nodes  were  originally  introduced  in  (21],  and  we 
generalized  them  in  our  method-level  IPR.  The  operational  semantics  for  the  nodes 
in  the  method-level  IPR  are  given  using  the  Petri-net  by  defining  the  functionality 

and  copy,  can  be  represented  as  a simple  net  as  shown  in  Figure  4.4.  The  input  places 
indicate  the  availability  of  the  input  value  to  the  function  and  the  output  places 
indicate  the  availability  of  the  result  of  the  function  application.  The  semantics  of 


Figure  4.4.  The  semantics  of  the  function  application 
the  control  function  nodes,  such  as  selector,  distributor  and  merge,  and  list  handling 
nodes,  such  as  construct  and  split,  are  represented  by  the  Petri-net  in  Figures  4.5-4.7. 
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Figure  4.6.  The  operational  semantics  of  the  distributor  node 
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Figure  4.7.  The  operational  semantics  of  the  list  handling  nodes:  (a)  construct  node 
(b)  split  node  (c)  merge  node 


In  this  section,  we  present  the  transformation  rules  in  the  two  levels.  The  PROOF/L 


program  is  first  transformed  into  the  IPR  and  then  the  IPR  is  transformed  to  the 
target  code.  The  former  is  called  front-end  transformation  and  the  latter  back-end 
transformation.  The  front-end  transformation  is  to  make  all  the  parallelism  explicitly 
expressed  in  the  IPR.  This  step  is  a semantics-oriented  transformation  and  machine 
dependent  issues  are  not  involved  at  all.  The  back-end  transformation  is  performance- 
oriented  and  the  machine  dependent  parameters,  such  as  communication  overhead, 
number  of  links  and  number  of  processors,  are  used  to  perform  various  analyses.  In 
the  following  the  transformation  rules  from  the  PROOF/L  program  to  the  IPR  and 
the  transformation  rules  from  the  IPR  to  a target  code  are  given.  We  also  show  that 
the  transformation  from  the  PROOF/L  program  to  the  IPR  preserves  the  correctness 
of  the  original  PROOF/L  program. 

Hi RepBffitntation  of  Interactions  among  Objects 

We  transform  the  behavior  of  objects  into  an  IPR.  The  behavior  of  an  object  is 
specified  in  the  body  of  the  object.  However,  the  bodies  are  defined  only  for  active 
and  pseudo-active  objects,  not  for  passive  objects.  The  absence  of  the  bodies  for 
the  passive  objects  causes  inconvenience  for  giving  semantics  for  the  passive  objects. 
For  detailed  information  regarding  these  constructs,  refer  to  [82].  To  overcome  this 
drawback,  we  define  the  interactions  among  objects  using  the  following  constructs: 

- SEQucncial  execution  of  methods:  When  the  methods  mt,  mi, . . . , m„  are  executed 
sequentially  in  the  order  mi,  mi, ....  m„,  its  behavior  is  specified  as  SEQ(mi,  m2, . . . ,rn„) 

- CONcurrent  execution  of  methods:  When  the  methods  mi,  m2, ....  m,  are  executed 
concurrently,  its  behavior  is  specified  as  CON(mi,m2, .. , ,m„) 

- WAIT  for  method  invocation:  When  an  object  is  waiting  for  the  invocation  of  its 


method  m by  another  object  0 to  proceed  with  its  execution,  its  behavior  is  specified 
as  WAlT(m,0) 

- SELect  a method  for  execution  based  on  a condition:  SEL  construct  behaves  like 

the  CASE  statement  in  ordinary  programming  languages.  When  an  object  selects 
one  of  the  methods  for  execution  from  among  the  methods  based  on 

a condition,  its  behavior  is  specified  as  SEL(mi,mj,...,m„) 

- ONE-OF  the  methods  for  execution  from  a group  of  possible  methods:  ONE- 
OF  construct  is  used  in  cases  where  different  objects  could  try  to  invoke  the  meth- 
ods defined  in  the  object  0 simultaneously.  The  object  0 permits  only  one  ob- 
ject to  invoke  its  method  at  a lime.  This  construct  serializes  the  requests  and 
is  typically  used  to  describe  the  behavior  of  the  shared  writable  objects.  Note 
the  difference  between  the  SEL  and  the  ONE-OF  construct.  Among  the  set  of 
methods  mi, . . . ,m„,  defined  in  an  object,  when  the  object  permits  only  one  of  its 
methods  to  be  invoked  by  other  objects,  the  behavior  of  the  object  is  specified  as 
ONE  - 0F(H'A/T(m,,0,)1 ....  WAIT(mn,  Os)). 


We  present  the  front-end  transformation  rules  on  two  levels:  object-level  and 
method-level. 

Object-level  transformation 

Using  the  Petri-net  defined  for  the  object-level  IPR,  we  define  the  transformation 
rules  used  in  the  object-level  transformation.  To  do  so,  we  define  the  compositional 
semantics  of  the  operators  used  in  expressing  the  behavior  of  the  objects  with  the 


Petri-net  as  follows: 


®, 


Wi  = {fl.Ti.fi, /i,0,} 

Nt  = {P,,T„F>,  1,.D,) 

in  which  fl,7i,fl>A  and  D„i  =1,2,  arc  disjoint  unless  specified 


PH  = (P,  U fl,T.  U T,  U <,fl  U fl  U D,  X {1}  U {<)  X /„  /„  D,} 


•j  Oj 


Figure  4.9.  The  semantics  of  the  sequential  invocation 
DfUnilion  1.3.3  The  transformation  rule  for  CON(e,,e,)  is  defined  as  a net 

VM  = {P,  U Pi  U p,  T,  U r,  U I,  F,  U Ft  U (p,  t ) U (f } 
x(/iU/s).(p}.OiUOs}. 

The  CON  representation  requires  an  addition  of  a place  p and  a transition  t in  order 
to  allow  the  concurrent  execution  of  the  two  processes  represented  by  each  net.  The 
concurrent  composition  of  the  two  nets  is  shown  in  Figure  4.10. 

The  transformation  rule  for  SEL  is  given  in  the  following  definition. 

Definition  1.3. i The  transformation  rule  for  SELfe-1,  e-ij  is  defined  as  a net 
VM  = {Pi  U Ps  U p, T,  U T, U <i  U Ij,  Fi  U F,  U (p,  li)U 

(P.l3)  U {<.}  » /,  U {<,}  x l„  {p},  D,  U Ds}. 

The  SEL  representation  requires  an  addition  of  a place  p and  two  transitions  fj  and 
Ij  so  that  only  one  transition  can  be  fired  each  time.  The  composition  of  the  two 
nets  for  selective  invocation  is  given  in  Figure  4.11. 

The  transformation  rule  for  ONE-OF  is  given  in  the  following  definition. 
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Definition  i.3.5  The  transformation  rule  for  OSE-OF(ci,tj]  is  defined  as  a net 
Ptf  = {Pi  U Pj.T’i  U Tj,  F,  U F,,  /,  U /,,  D,Ufl,| 
in  which  | /*i  U ft  |=|  | + | ft  | — 1. 

Note  that  there  is  a common  place  between  the  two  nets  Ni  and  jVj.  The  ONE- 
OF  construct  is  used  to  represent  mutually  exclusive  access  to  the  shared  object. 
Such  access  is  specified  using  the  pseudo-function  72  in  the  body  of  the  object.  To 
represent  the  mutually  exclusive  access  to  the  shared  object,  a special  place,  called  the 
bottleneck  place,  is  associated  with  the  transition  representing  the  method  invocation 
which  results  the  modification  of  the  shared  object.  The  bottleneck  place  for  the 
object  is  unique,  and  thus  the  number  of  places  after  the  composition  of  the  two  nets 
is  reduced  by  one.  This  composition  is  called  fusion  of  nets  via  place,  and  is  shown  in 
Figure  4.12.  The  transformation  rules  presented  so  far  show  only  the  composition  of 
the  two  nets,  but  these  rules  can  be  easily  generalised  in  the  case  of  the  composition 
of  more  than  two  nets. 
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Ptf={P>VP„T,U T„ F,  U F„ I,  U /„ D,  U D, } 

in  which  | r,  u r,  |=|  r,  | + 1 r,  | -i, 


71 


Figure  4.12.  The  semantics  of  the  mutually  exclusive  access 
in  advance  to  the  caller  object,  and  the  called  object  also  knows  of  the  existence  of 
the  caller  object.  In  the  behavior  of  the  caller  object,  the  method  name  is  given 
with  its  called  object.  In  the  behavior  of  the  called  object,  the  WAIT  construct  is 
explicitly  used  to  specify  the  possible  communication.  In  other  words,  the  existence 
of  communication  between  the  two  objects  can  be  detected  by  examining  the  Petri- 
net  representation  for  each  body  of  the  two  objects.  The  two  nets  are  composed  via 
the  common  transition  ( as  shown  in  Figure  4.13.  This  composition  of  nets  is  called 

During  the  transformation,  several  transitions  need  to  be  added  to  compose  the 
nets.  These  transitions  are  not  introduced  to  add  functionality  to  the  original  nets, 
but  added  for  the  purposes  of  the  composition  only.  In  the  following  we  distinguish 


Figure  4.13.  The  semantics  of  the  communication  between  two  objects 


these  additional  transitions  and  the  transitions  corresponding  to  the  functionality  of 
the  program. 


The  useful  transition  is 


transition  to  represent  the  method  in 


Definition  1.3.8  The  dummy  transition  is  a transition  used  [or  composition  0/  the 


In  the  transformation  rules,  the  transitions  introduced  to  compose  the  original 
nets  are  all  dummy  transitions.  The  firing  of  the  dummy  transition  does  not  affect 
the  object  slate. 

Method-level  transformation 

We  present  the  transformation  rules  from  the  PROOF/L  method  definition  to  the 
method-level  IPR.  In  the  PROOF  computation  model,  each  method  consists  of  an 
optional  guard  and  an  expression.  Before  the  expression  is  executed,  the  synchroniza- 
tion constraints  specified  by  the  guard  construct  should  be  satisfied.  We  present  the 


73 


semantics  of  the  guard  mechanism  in  the  Petri-net.  Because  each  method  is  repre- 
sented as  a transition  in  the  object-level  IPR,  the  semantics  for  the  guard  mechanism 
can  be  given  by  refining  a transition  using  the  refinement  rules  given  in  the  previous 
section.  The  guard  semantics  is  shown  in  Figure  4.14  in  which  the  representation  of 
the  transition  in  the  object-level  IPR  is  shown  as  the  two  dotted  transitions  and  the 
boundary  of  the  method  is  also  specified  as  a dotted  rectangle. 

The  transformation  rules  for  the  general  function,  functional  forms  and  control 
functions  are  shown  in  Figures  4.15  - 4.16.  In  Figure  4.15,  the  transformation  for  a 

function  /(ji(xi,ii-),jj(xa,*r) »»(*>., x„.))  is  shown  in  (a),  the  transformation 

for  the  functional  form  a f[x i ,X:, . - . ,x„]  is  shown  in  (b),  the  transformation  for  the 
functional  form  /?|/i,/a, . ..  ,/„][xi,xa, . . . ,xn]  is  shown  in  (c),  the  transformation  for 

the  functional  form  7(61,63 6„][xi,x3,  - . . ,x„]  is  shown  in  (d),  the  transformation 

for  the  functional  form  <[/i,/j,  . . . ,/„][x]  is  shown  in  (e)  and  finally  the  transforma- 
tion for  the  functional  form  q/(xi,X2,...,xn]  is  shown  in  (f).  In  Figure  4.16,  the 
transformation  for  the  while  (p  / ][x]  is  shown  in  (a)  and  the  transformation  for  the 
>f  I P 1 9 ][*)  i*  »h°wn  in  (b). 


Figure  4.14.  The  semantics  of  the  method  invocation  with  a guard 


Figure  4,15.  The  transformation  rules  for  function  and  functional  forms 
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Figure  4.16.  The  transformation  rules  for  the  control  functions 
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In  this  section,  we  show  that  the  front-end  transformation  is  a correctness  pre- 
serving transformation.  In  order  to  show  the  correctness  preserving  transformation, 
we  give  an  interleaving  semantics  to  the  PROOF/L  program  by  defining  a labeled 
transition  system  and  a compositional  Petri-net  semantics  which  can  be  obtained  by 
applying  the  front-end  transformation  rules  to  the  PROOF/L  program.  Then  we 
show  the  interleaving  semantics  for  the  PROOF/L  program  can  be  retrieved  from 
the  Petri-net  representation  of  the  corresponding  IPR  form.  Note  that  the  Petri-net 
is  used  to  not  only  give  the  semantics  for  the  PROOF/L  programs  but  also  to  give 
formalism  to  object-level  representation  in  the  IPR.  In  the  interleaving  semantics,  a 
step  of  a single  process  is  described  as  a step  of  the  whole  system,  changing  some 
global  state.  Parallel  behavior  of  the  system  is  specified  in  a sequential  interleaved 
manner.  Thus,  a computation  or  program  execution  is  defined  as  a sequence  of  such 
global  steps.  To  give  the  interleaving  semantics  for  the  PROOF/L  program  we  use  a 
labeled  transition  system  [51]  as  an  abstract  machine. 

Delinition  1.3.9  A labeled  transition  system  is  a structure  A such  that 


An  element  ( s,a,s ')  g-»  is  called  a transition  (labeled  with  the  action  a)  from  a state 
s to  another  state  s by  a step  a,  and  will  be  written  as  s — * a . 


A = (S, -*,/o) 


S is  a set  of  states, 

sCSxActxSisa  transition  relation 

in  which  Act  is  an  action  fo  change  the  state,  and 


la  is  an  initial  state. 


In  the  following,  we  define  a PROOF/L  program  as  a set  of  processes  and  then  give 
the  semantics  to  it.  Each  process  corresponds  to  the  body  of  the  non-passive  object. 
We  can  associate  a set  of  local  states,  called  Si,  to  each  object  Oi.  Then,  the  state  of 
the  PROOF/L  program  can  be  represented  as  a Cartesian  product  S,  x Si  x . . . x S„. 
The  actions  to  change  stales  are  represented  by  constructs,  such  as  SEQ,  CON,  ONE-OF, 
SEL  and  WAIT.  Using  the  labeled  transition  system  defined  in  the  Definition  4.3.9  and 
the  interpretation  of  the  PROOF/L  program  given  above,  the  interleaving  semantics 
of  the  PROOF/  L program  can  be  defined  as  follows: 

Definition  1.3.10  An  interleaving  semantics,  Aj,  for  the  PROOF/L  program,  PROG, 
can  be  represented  by  the  following  labeled  transition  system: 

Ai{PROG)  ■■=  (S,-,/o) 

S is  a Cartesian  product  Si  x & x ...  x S„, 

/o  is  an  initial  state , and 
-CSxAclxS 

in  which  the  transition  relations  are  generated  by  the  following  rules. 

In  the  rules  presented  below,  ^ means  that  if  P is  true  then  Q is  also  true. 

Rule  I:  Sequential  method  invocation 

Rule  2:  Unconditional  selective  method  invocation 


ONE  - OF(e,,ej)  A e,.,ONE  - OF(e,,e,)  A e,. 


Rule  3:  Conditional  selective  method  invocation 


SEL(e,,e2)  A ei',SEL(e2,ei)  A «|> 


Rule  4:  Parallel  method  invocation  without  communication 


CON(e,,ea)  A CON(e,.,ea),CON(ea,e,)  A CON(e2,e,.) 


Rule  5 Parallel  method  invocation  with  communication 


CON(e„ea)  CON(e,.,ea.) 


The  set  S of  PROOF/L  terms  is  defined  by  the  following  production  system, 
e ::=  nil  | SEQ(e,  e)  | CON(e,  e)  | ONE  - OF(e,  e)  | e..„„(., 

in  which  a € Act  and  eaiUtan(a)  means  the  communication 
between  two  processes  via  the  action  labeled  a. 

In  the  following  we  prove  that  the  front-end  transformation  preserves  the  cor- 
rectness of  the  PROOF/L  program  by  showing  that  an  interleaving  semantics  of  the 
PROOF/L  program  is  retrievable  from  its  corresponding  IPR. 

Theorem  i.3.1  The  transformation  rules  given  by  Definitions  f.3.1  - f. 3.6  preserve 
the  meaning  of  the  bodies  of  the  PROOF/L  program. 

Proof  ol  Theorem  i.3.1  The  proof  of  this  theorem  consists  of  the  following  two  steps: 
I)  The  Petri-net  semantics  given  for  the  bodies  of  the  PROOF/L  program  can  be 


vied «"(« “ lMcd  tnnsi,ion 

t)  ftvm  the  labeled  transition  system  obtained  in  step  1),  the  interleaving  semantics 
/lir  Ihr  Original  PROOF/L  program  can  be  retrieved. 

HV  rail  a PROOF/L  program  as  PROS,  and  let  its  IPR  representation  in  the  Petri- 
p/f(PHOG):=(P,T,F,l,D)-  We  first  build  a labeled  transition  system  CIS 
fromPtf  as  follows: 

ClS{  PnOC)  can  be  defined  as  follows: 

CTS(PROQ)  :=  {SAP),  I) 

S+( p)  ^ a set  of  non-empty  subsets  of  a set  P, 

SAP)  x Act  x SAP)  Is  a transition  relation,  and 

s±s,seSAP),>  e SAP)  if  “d  only  if  3<eT 
Vpi.Vpj  such  that  p,  x ( 6 F and  t x p,  € F.numfp,)  > 0. 
The  transition  results  in  Vpj,num(pj)  is  increased  by  one. 

ItV  show  that  from  CTS(PROG ) an  interleaving  semantics  of  the  PROG  is  re- 
movable by  showing  that  l)  for  each  useful  transition  in  PM  there  is  a corresponding 
action  in  CIS  and  S)  the  synchronization  among  objects  in  the  PROG  an  not  vio- 
lated- 

Pirsl  we  show  that  then  an  only  a finite  number  of  dummy  transitions  introduced 
during  the  transformation  of  any  PROOF/L  program.  For  each  transformation,  then 
an  at  most  two  dummy  transitions  added.  Thus,  for  a finite  number  of  the  bodies  in 
any  PROOF/L  program,  we  only  need  to  introduce  a finite  number  of  dummy  transi- 
tions. The  dummy  and  the  following  useful  transitions  can  be  transformed  to  a new 


useful  transition  without  changing  the  meaning  of  the  original  net.  Thus,  we  have 
proved  I). 

Next,  from  the  definition  of  the  transformation  rule  for  communication  between 
two  bodies,  we  know  that  a transition  corresponding  to  the  communication  cannot 
occur  unless  both  the  two  objects,  the  caller  and  the  called,  are  not  ready  to  commu- 
nicate. Both  the  caller  and  called  objects  can  proceed  only  after  the  two  objects  are 
synchronised  for  communication  ( method  invocation).  We  also  know  by  examining 
the  transformation  rules  given  in  Definitions  f.S.I  - f.S.f  that  the  transformation 
rules  do  not  violate  the  method  invocation  sequences  of  the  original  PROOF/L  pro- 
gram. Since  these  transformation  rules  with  the  rule  for  communication  are  the  only 
rules  used  for  the  transformation,  the  synchronisation  amo ng  the  objects  cannot  be 
violated.  Therefore,  we  have  proved  2). 

In  conclusion,  from  I)  and  2),  if  we  give  a total  ordering  to  the  transitions  in  CIS, 
the  total  ordering  corresponds  to  an  interleaving  semantics  of  the  original  PROOF/L 
program. 

In  the  following,  we  show  the  method-level  transformation  also  preserve  the  cor- 
rectness of  the  method  definition  in  the  PROOF/L  program. 

Theorem  1.3.2  The  transformation  rules  for  method-level  in  the  IPR  preserve  the 
meaning  of  the  methods  given  in  the  original  definition  in  the  PROOF/L  program. 
Proof  of  Theorem  1.3.2  In  the  PROOF  computation  model,  each  method  is  defined  as 
an  applicative  function  without  side-effects  within  the  method.  The  relations  among 
the  statement  in  each  method  definition  are  based  on  data-depcndency  and  no  explicit 
communication  is  involved.  Thus,  it  is  sufficient  to  show  that  the  method-level  trans- 

the  method  definition.  Since  any  node  in  the  method-level  IPR  can  be  regarded  as  a 
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function,  by  examining  the  transformation  rule  for  the  general  function,  it  is  obvious 
that  no  violation  of  data  dependency  relationship  exists.  Therefore,  we  show  that  the 
meaning  the  method  in  the  PROOF/L  program  can  be  preserved  in  the  representation 
of  the  method  in  the  IPR . 

We  can  also  prove  theorem  4.3.2  by  building  a labeled  transition  system  in  which  no 
explicit  communication  is  involved  and  showing  that  an  interleaving  semantics  can 
be  retrieved  from  the  semantics  of  the  method-level  representation  of  the  IPR. 

Theorem  J.S.S  The  transformation  rules  we  introduced  for  translation  of  the  PROOF/L 
program  to  its  corresponding  IPR  preserve  the  meaning  of  the  PROOF/L  program. 

Proof  of  Theorem  J.S.S  From  Theorems  4-3.1  and  f.S.S,  it  is  proved. 

Lemma  J.S.I  The  transformation  by  the  transformation  rules  given  in  Definitions 
f.S.I  - f.S.d  preserves  the  correctness  of  the  original  PROOF/L  program. 

Proof  of  Lemma  J.S.I  By  the  Theorem  4-3.3,  the  correctness  of  the  PROOF/L  pro- 
gram is  preserved  in  the  IPR. 

4.3.4 Back-end  Transformation 

Once  the  IPR  is  generated  from  the  PROOF/L  program,  the  various  analyses  are 
done  in  the  IPR  and  the  modified  IPR  is  generated.  The  analyses  includes  grain  size 
determination  and  clustering,  which  will  be  explained  in  the  next  chapter.  In  this 
section,  the  translation  rules  from  the  IPR  to  a given  target  language  are  given. 

For  the  body  of  each  object,  the  IPR  is  given  as  a marked  Petri-net  in  the  front-end 
transformation.  In  the  back-end  transformation,  the  marked  Petri-net  for  each  body 
can  be  translated  to  a local  scheduler  in  each  processor.  A process  can  be  created  for 
each  transition  in  the  net  and  the  local  scheduler  executes  the  processes  in  a manner 
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that  the  original  dependency  relationships  arc  not  violated.  In  the  case  of  a network 
of  Inmos  transputers  in  which  the  allocation  information  of  the  processors  should  be 
known  prior  to  run-time,  the  local  scheduling  information  obtained  from  the  object- 
level  IPR  needs  to  be  embedded  into  the  target  code.  Since  the  net  structure  can  be 
partitioned,  it  is  not  necessary  to  restrict  the  boundary  of  a scheduling  unit.  That  is, 
an  object  can  be  partitioned  into  a set  of  parallel  processes,  and  a number  of  objects 
can  be  grouped  together  to  form  a larger  process.  In  either  case,  the  net  structure 
can  be  partitioned  or  grouped  accordingly. 

A lock  manager  needs  to  be  associated  to  each  shared  object  at  this  stage.  The 
lock  manager  controls  access  to  the  shared  object.  In  the  IPR,  the  existence  of 
the  lock  manager  is  implicit  in  the  sense  that  the  complete  semantics  of  the  lock 
manager  is  hidden.  During  the  back-end  transformation,  a proper  concurrency  control 
mechanism  can  be  associated  to  each  shared  object. 

For  each  primitive  function  node  in  the  method-level  IPR  and  control  functions, 
we  briefly  describe  the  general  transformation  rules  which  are  not  directly  targeted 
to  any  specific  target  language.  To  introduce  general  translation  rules,  we  select 
basic  language  constructs  available  in  the  existing  programming  languages.  These 
constructs  include  assignment  statement (:=),  while  loop  s(oIemen/(while  ..  do), 
conditional  statement(il  ..  then  ..  else),  case  sfafemenlfswitch  ..  case  1 ... 
cose  n),  function  s/atemen/(function(input,output))  and  process  creation  state- 
ment(  create(process)). 

In  the  IPR,  every  node  can  be  regarded  as  a function  which  receives  input  pa- 
rameters and  which  returns  a value  as  a result  of  function  application.  The  function 
can  be  translated  to  a simple  function  or  a process  according  to  the  following  rule: 
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Rule  Bl:  Function  translation 

Let  func  be  a function.  Then  func  is  translated  either  as  a single  assignment  state- 
ment or  a process  creation  statement. 
ruleBl.l:  a primitive  function 

0:=/unc(/i,  /*) 

in  which  if  func  represents  primitive  operators,  such  as  arithmetic 


0:=  operator  /|  when  n = 1, 

0:=/j  operator  Ij  when  n = 2. 
rule  B1.2:  a user  defined  function 

create(/une(/|, . . . O)) 

The  nodes  to  which  the  Rule  Bl  can  be  applied  include  function,  id,  constant , copy 

Control  function  nodes,  such  as  selector  and  distributor,  are  translated  according 
to  the  following  rule. 


Rule  B2  Control  function  translation: 
B2.1:  selector 


if  c = true  then 


case  m:  0:=/„ 

B2.2  distributor 

if  c=  true  then 

0,:=/ 


when  m > 2 

case  1:  0,:=/ 
case  2:  0,:=/ 

case  m:  0m:=l 

split,  construct  and  merge  nodes  are  used  in  the  transformation  rules  for  various 
functional  forms  as  shown  in  Figure  4.15.  In  addition  to  these  transformation  rules, 
to  utilise  the  high-level  programming  language  constructs,  control  structures,  such 
as  while  loop  and  if-then-else  can  also  be  identified  in  the  1PR. 


Using  the  transformation  rules  given  in  this  chapter,  various  programs  have  been 
written  in  PROOF/L  and  transformed  via  the  two-step  translation  to  the  target 
programs  written  in  Inmos  C for  the  execution  of  the  Inmos  transputer  network. 


The  example  programs  include  factorial,  producer-consumer  problem,  dining  philoso- 
phers problem,  warehouse  management  systems  and  air-base  defense  systems.  The 
PROOF/L  codes,  and  the  1PR  codes  for  the  selected  examples  are  given  in  the  Ap- 
pendix A2. 


ALLOCATION  OF  PROOF/L  PROGRAMS 


One  of  the  reasons  we  execute  a program  in  a parallel  processing  system  is  to  re- 
duce the  time  required  for  completion.  It  would  scein  that  a simple  solution  would  be 
to  detect  all  the  parallelism  in  the  program  and  execute  it  in  parallel  on  the  parallel 
processing  system.  However,  this  simple  solution  does  not  work  in  most  cases.  As 
the  number  of  parallel  processes  increases,  the  communication  between  processes  also 
increases.  Thus,  the  parallelism  needs  to  be  controlled  so  that  the  gain  of  parallel 
execution  cannot  be  overshadowed  by  the  communication  overheads  due  to  excessive 
parallelism.  We  identify  this  controlling  parallelism  problem  as  an  allocation  prob- 
lem. Various  terms  used  in  the  literature,  such  as  allocation,  partitioning,  clustering 
and  multiprocessor  scheduling , are  closely  related  each  other.  In  general,  task  alloca- 
tion is  used  in  distributed  computing  systems,  denoting  the  distribution  of  tasks  into 
a set  of  computers.  Partitioning  or  clustering  is  to  group  tasks  into  a set  of  partitions 
or  clusters,  multiprocessor  scheduling  is  to  schedule  tasks  to  reduce  the  completion 
time.  We  consider  task  allocation  as  a general  term  which  also  includes  the  meaning  of 
clustering,  partitioning  and  multiprocessor  scheduling.  Whenever  further  distinction 
is  required,  we  use  proper  terms  based  on  the  definitions.  This  chapter  is  organized 
as  follows:  In  Section  5.1,  we  present  the  existing  task  allocation  approaches,  discuss 


their  limitations  and  briefly  outline  our  approach.  In  Section  5.2,  we  present  a model- 
ing method  for  representing  the  PROOF/L  program  as  a directed  graph  and  a simple 
bottom-up  clustering  strategy  and  illustrate  it  with  an  example.  In  Section  5.3,  we 


illustrate  the  importance  of  the  grain  size  determination,  present  strategies  for  pipe- 
lined parallelism,  tree  parallelism  and  graph  parallelism,  and  compare  them  with  the 
existing  approaches. 

5.1  Task  Allocation  Approaches 

The  problem  of  task  allocation  in  a program  execution  in  distributed  or  parallel 
processing  systems  has  been  studied  by  many  researchers,  and  the  existing  approaches 
can  be  divided  into  two  categories:  task  allocation  without  precedence  relations  and 
task  allocation  with  precedence  relations.  Task  allocation  without  precedence  rela- 
tions among  tasks  can  be  further  divided  into  three  sub-categories:  graph  theoretic, 
mathematical  programming  and  heuristic. 

In  [74],  Stone  introduced  a graph  theoretic  approach  for  the  task  allocation  prob- 
lem in  the  case  of  two  processors.  The  program  is  regarded  as  a set  of  tasks,  each 
of  which  communicates  with  other  tasks.  In  the  program  graph,  a node  represents 
a task,  and  an  edge  between  two  nodes  represents  the  existence  of  communication 
between  them.  Then  the  network  flow  algorithm  is  applied  to  this  graph.  While  it 
is  simple  to  obtain  an  optimal  solution  with  this  method,  it  can  be  applied  only  to  a 
limited  number  of  processors,  which  severely  limits  its  applicability.  An  extension  of 
this  method  for  the  cases  of  more  than  two  processors  has  been  proposed  in  a special 


a heuristic  solution  based  on  this  method  has  also  been  proposed  in  [55].  Another 
method  based  on  a graph  has  been  suggested  by  Shen  in  [72].  Shen  developed  a 
graph-matching  algorithm  to  obtain  a task  allocation  and  minimize  the  completion 
lime  of  the  program  when  precedence  relations  among  tasks  do  not  exist. 

The  mathematical  programming  approach  generally  involves  a 0-1  integer  pro- 
gramming technique  for  assigning  programs  or  data  files  to  processors  to  minimize 


the  given  cost  function  with  specific  constraints  [17, 25,  57,  80].  This  approach  allows 
constraints  to  be  incorporated  into  the  task  allocation  model,  but  is  limited  by  the 
amounts  of  the  time  and  space  required  because  the  time  and  the  space  complexities 
grow  as  exponential  functions. 

While  these  approaches  require  exponential  time  and  space  to  obtain  optimal  solu- 
tions, heuristic  approaches  can  obtain  solutions  efficiently  by  sacrificing  the  optimality 
of  the  solution.  A popular  heuristic  approach  is  the  clustering  method  [24, 37, 68, 69]. 
In  [24],  the  proposed  method  consists  of  two  phases.  First,  a task  clustering  algo- 
rithm is  applied  to  minimize  interprocessor  communication.  Second,  to  balance  the 
load,  underloaded  and  overloaded  processors  arc  identified  and  adjusted.  In  [37,  69], 
a pair  of  tasks  are  searched  for  clustering  such  that  their  clustering  eliminates  the 
greatest  possible  interprocessor  communication  cost.  This  process  continues  until  all 
the  pairs  are  clustered.  In  [68],  two  heuristic  algorithms  have  been  presented.  One 
is  an  iterative  assignment-improvement  algorithms.  Any  initial  allocation  of  tasks  is 
transformed  by  reassigning  tasks  to  obtain  a better  allocation.  The  second  algorithm 
is  a clustering  method. 

In  the  three  approaches  we  have  discussed  above,  the  precedence  relations  among 
tasks  are  ignored.  When  there  are  precedence  relations  among  tasks,  the  goal  of  task 
allocation  is  to  reduce  the  completion  time.  In  general,  the  task  allocation  problem  for 
minimizing  the  completion  time  is  known  as  the  multiprocessor  scheduling  problem. 
The  multiprocessor  scheduling  problem  has  been  known  as  an  NP-complete  problem 
except  in  a few  very  restricted  cases  [32,  81].  There  have  been  many  heuristics 
proposed  to  obtain  efficient  solutions.  Among  them  list  scheduling  [1]  has  been 
popular  because  of  its  simplicity  and  sub-optimality.  In  list  scheduling,  the  program  is 
represented  as  a task  precedence  graph  in  which  each  node  represents  a task  and  each 
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assigns  a priority  to  each  of  the  tasks  and  places  tasks  in  an  ordered  list  according 
to  the  priority.  When  any  of  the  processors  is  ready  to  execute,  a task  with  the 
highest  priority  is  chosen  to  be  executed.  Thus,  the  core  of  list  scheduling  is  how  to 
determine  the  priority  of  each  task. 

In  [50],  the  path  having  the  longest  length  from  the  entry  vertex  to  the  exit  vertex, 
called  the  critical  path,  is  given  the  highest  priority,  and  each  of  the  remaining  tasks  is 
given  a priority  based  on  the  number  of  successors  to  each  task.  In  [67],  WP  heuristic 
has  been  presented.  WP  heuristic  first  gives  high  priority  to  tasks  that  compose  the 
critical  path.  For  the  rest  of  the  tasks,  composite  priorities  are  calculated  based  on 
three  criteria:  tasks  having  the  longest  execution  time  first,  tasks  having  the  largest 
number  of  successors  first,  and  tasks  having  largest  successor-tasks  first.  Although 
these  heuristics  are  known  to  be  successful,  the  limitation  is  that  the  communication 
cost  between  tasks  has  been  completely  ignored.  In  Kim  [52],  a heuristic  approach, 
called  linear  clustering,  has  been  introduced  in  which  not  only  execution  time  but 
also  communication  cost  are  considered  in  scheduling  a set  of  tasks  with  precedence 
relations.  Hwang[44)  introduced  a greedy  algorithm  called  ETF(Earliest  Task  First) 
in  which  communication  times  between  tasks  are  considered.  ETF  adopts  the  simple 
heuristic:  the  earliest  schedulablc  task  is  first  scheduled  to  an  idle  processor.  However, 
these  two  approaches  are  completely  dependent  on  the  grain  sizes  determined  by 
programmers  and  thus  the  performance  of  these  approaches  may  not  be  good  enough 
for  some  applications  in  which  the  grain  size  is  very  small. 

In  addition,  these  approaches  are  not  directly  applicable  to  allocate  the  programs 
based  on  the  computation  model  PROOF  due  to  the  following  properties: 


PROOF  offers  various  granu 
level. 


The  accesses  to  the  shared  object  need  to  be  mutually  exclusive. 


To  exploit  parallelism  in  different  granularity  levels  naturally  implies  a multi-level 
approach  in  which  a different  strategy  for  each  level  can  be  used.  The  problem  with 
such  a multi-level  approach  is  that  at  the  highest  level1  it  is  difficult  to  obtain  ap- 
propriate information  for  the  task  allocation.  In  the  PROOF  programs,  for  instance, 
accurate  execution  time  at  the  object  level  is  very  difficult  to  predict  because  (a)  we 
do  not  know  how  much  parallelism  can  bo  exploited  within  the  methods,  and  (b)  the 
execution  times  of  the  methods  are  dependent  on  input  data.  Another  problem  is 
that  the  access  to  the  shared  data  may  need  to  be  synchronized.  However,  in  most  of 
the  existing  allocation  approaches,  the  dependency  relationship  among  tasks  is  based 
on  data  dependency,  and  thus  synchronization  requirements  cannot  be  specified. 

In  order  to  allocate  programs  based  on  the  computation  model  PROOF,  we  use 
a two-level  allocation  approach.  At  the  object-level,  the  parallelism  among  objects 
i.e.,  coarse  grain  parallelism,  is  exploited.  The  objective  of  object-level  allocation  is 
to  group  the  objects  into  a set  of  dusters  so  that  communication  overhead  can  be 
reduced  while  keeping  the  potential  parallelism  among  the  objects.  Once  the  objects 
are  clustered,  each  cluster  has  at  most  one  active  object.  Thus,  it  is  very  likely  that 
in  each  cluster  there  is  one  object  busy  in  execution  at  the  moment.  We  use  as  input 
the  object  invocation  relations  which  can  be  obtained  from  the  object  decomposition 
phase.  At  the  method  level,  the  parallelism  within  each  method,  i.e.,  fine  grain 
parallelism,  is  exploited.  At  this  stage,  the  proper  grain  sizes  arc  determined  within 
each  method  by  analyzing  the  execution  and  communication  times.  Depending  on 
the  types  of  the  parallelism,  different  strategics  arc  required  to  meet  the  specific 
requirement.  We  present  grain  size  determination  strategies  for  the  three  patterns  of 
parallelism:  pipe-lined  parallelism,  tree-parallelism  and  graph  parallelism.  The  grain 
'largest  granularity  level 


size  determination  strategies  can  also  be  used  at  the  object-level  if  the  information 
about  the  execution  and  communication  times  is  explicitly  available  at  this  level. 


The  objective  of  object  partitioning  is  to  partition  a set  of  interacting  objects 
so  that  the  communication  cost  among  processors  can  be  reduced  while  keeping  the 
potential  parallelism  among  them.  The  input  for  our  algorithm  is  the  behavior  of 
the  objects  in  the  software  system  specified  using  the  constructs, such  as  SEQ,  CON, 
ONE-OF,  SEL  and  WAIT.  The  output  of  our  algorithm  will  be  a set  of  object  clusters 
and  their  communication  dependency  relations.  The  approach  consists  of  the  two 
stages:  a modeling  stage  and  a clustering  stage. 


We  begin  with  modeling  the  software  by  a directed  graph.  Each  object  is  repre- 
sented by  one  node  in  the  graph,  and  there  is  an  edge  between  two  nodes  if  and  only 
if  there  is  a precedence  relation  between  the  methods  in  the  two  objects.  Each  edge 

tween  the  two  nodes.  The  communication  weight  associated  with  an  edge  represents 
the  communication  cost  incurred  if  the  two  objects  represented  by  the  nodes  incident 
to  that  edge  communicate  with  one  another,  but  they  are  not  allocated  on  the  same 
processor. 

The  software  system  is  modeled  by  a weighted,  directed  graph  G = (V,  E ).  The 
graph  G = (V,  E)  has  a set  of  nodes  V and  a set  of  edges  E such  that: 


The  ith  object  is  represented  by  a node  o,  in  V. 


• An  edge  (o„o>)  is  in  E if  there  is  a precedence  relation  between  the  two  objects 
such  that  an  invocation  of  a method  m j defined  in  o,  is  followed  by  an  invocation 
of  a method  m,  defined  in  Oj. 

. A communication  weight  w.j  is  associated  to  every  edge  («,«)- 
We  assume  that  the  communication  weights  can  be  obtained  by  analysing  the 
requirement  specification.  The  factors  used  to  determine  the  weights  can  include  the 
frequency  of  the  method  invocations  and  the  amount  of  data  transfer  required  for 
each  invocation. 

The  PROOF/L  program  is  modeled  as  a directed  graph  using  the  following  rules. 
For  the  simplicity,  we  use  the  weight  T for  each  possible  invocation  when  we  derive 
the  weights  in  the  following  rules. 

Rule  X.  Oi  : C0N(oj,O3,.  ...o„)  describes  a case  where  the  objects  oj.oj,. . 
and  On  are  executed  concurrently  after  being  invoked  by  the  object  Oi . It  corresponds 
to  a subgraph  G.  = (V„  E.)  where 

V,  = (0|,0j o„), 

E.  = {(«,.«<),  2<i<")- 

The  communication  weights  arc  assigned  to  the  edges  as  follows: 

1.  If  (0|,0j)  is  new,  u>i,  = 1. 

2.  If  (oi.Oj)  is  old,  new  w,j  = w,j  + 1 

Rule  2.  0|  : SEQ(o,,os,. ..  ,o„)  describes  a case  where  oi  invokes  Oj,os,. ..  ,0,-1, 
and  o„  in  a sequential  order.  It  corresponds  to  a subgraph  G.  = (Vi,  E.)  where 

Vi  = {oi,oj o.), 

E,  = {(o,-,Oi+i),  1 < « £ n - !)• 

The  communication  weights  arc  assigned  to  the  edges  as  follows: 


!•  If(0i,0iti)ia 


2.  If  (o„ortl)  is  old,  new  Wij  = + I. 

Rule  3.  0|  : ONE-OF(oj, 03, • • ,o„)  and  o,  : SEL(  02,03, . . . , o„)  describe  a case 
where  01  invokes  only  one  oi , 2 < j <n.  Both  correspond  lo  a subgraph  G,  = (V, , E,) 


£.  = {(o..o>)>  2 <;•<»}. 

The  communication  weights  in  E , are  assigned  to  the  edges  as  follows: 

1.  If  (o„0|)  is  new,  u>,,  = l/(n  - I). 

2.  If  (<>! , Oj)  is  old,  new  ui,,  = in,,-  + l/(n  - 1). 

Note  that  instead  of  the  actual  method  names  the  corresponding  object  names  are 
used  in  these  rules.  The  above  three  rules  can  handle  simple  clauses  in  which  only 
one  construct  is  used  to  specify  the  behavior  of  the  object.  The  models  using  these 
rules  are  illustrated  in  the  Figure  5.1. 

In  the  following,  we  present  a rule  that  can  handle  nested  clauses. 

Rule  4.  It  is  applied  when  nested  clauses  are  used  to  specify  the  object  behavior. 
The  steps  are: 

1)  Modify  the  object  behavior  by  substituting  all  the  nested  clauses  with  new 

i)  Select  and  apply  a rule  based  on  the  construct  used.  For  every  new  object 
introduced  in  step  1),  do  the  following  steps: 
i.l)  Apply  an  appropriate  rule  and  preserve  the  edge  relationships  with 
other  objects. 


c)0| : SELorONE-OF<°i'°3  •°»* 


Figure  5.1.  The  modeling  of  the  object  behavior 
S.SJ  Assign  communication  weights  as  described  earlier  in  this  section. 

We  now  have  a graph  in  which  each  node  represents  an  object  (the  same  as  in 
the  initial  graph)  and  to  every  edge  there  is  one  weight  ui^  representing  the  commu- 

5.2.2  Clustering  in  the  Obiect-level 

Once  the  modeling  of  the  software  as  a directed  graph  is  done,  we  apply  a bottom* 
up  clustering  approach  to  the  graph.  The  main  objective  of  this  part  is  to  reduce 


the  unnecessary  communication  overhead  by  clustering  the  objects  while  keeping  the 
potential  parallelism  among  objects. 

The  input  is  a weighted,  directed  graph  G ~ ( V'\  £').  v'  = ...  ,vp}  where 

Vi  is  an  object  for  1 < » < p.  Every  edge  (uj,tq)  € E‘  has  a weight  u.',, . The  output 
is  a directed  graph  in  which  each  node  represents  a cluster  and  each  edge  represents 
the  communication  requirements  between  clusters. 

The  steps  we  take  in  clustering  the  nodes  of  the  graph  are  as  follows: 

Algorithm  S.S.I  Object  Clustering 
input:  d = (Y\E') 
output:  a set  oj  clusters 

Initially,  each  object  o,  is  a cluster  { o. } . 

For  each  object  node  o having  no  incoming  edge,  do  the  following: 

Step  1 Set  o os  current  object  o,. 

Step  2 Find  a node  o>>  such  that  ( o„ , o* ) is  in  E and  w9s  is  the  largest. 

Step  3 Cluster  jot)  and  the  cluster  which  o„  belongs  to. 

Step  4 Remove  the  edges  from  o*  to  the  other  nodes  and  from  the  other  nodes  to  oa. 
Step  5 If  the  node  oj  has  an  outgoing  edge  to  any  node  then 
Set  os  as  the  current  object  oB 
Go  to  Step  S 
else 

Stop 

In  Step  1,  a node  representing  an  active  object  is  set  to  a current  node  to  be 
processed.  In  Step  2,  a node  is  selected  so  that  the  clustering  of  that  node  with 
the  current  node  can  reduce  the  communication  overhead  most.  In  Step  3,  the  node 
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selected  in  Step  2 is  clustered  with  the  cluster  which  the  current  node  belongs  to. 
In  Step  4 the  other  nodes  connected  to  the  current  node  are  disconnected  from  the 
current  node.  This  step  guarantees  that  all  possible  parallelism  in  the  software  is 
retained  by  not  clustering  any  two  objects  having  the  possibility  of  parallel  execution. 
These  steps  arc  repeated  until  all  the  active  objects  arc  processed.  Because  each  edge 
needs  to  be  visited  at  most  once,  the  time  complexity  of  this  clustering  approach  is 
0(e)  where  e is  the  total  number  of  edges. 

The  result  of  the  object  clustering  is  a set  of  clusters,  each  including  at  most  one 
active  object  and  the  objects  invoked  by  that  active  object.  Thus,  no  parallelism 
among  objects  has  been  lost.  Once  the  clustering  of  the  objects  is  complete,  each 
cluster  is  considered  as  an  independent  sub-program  to  be  analysed  independently 
of  the  other  subprograms.  In  each  sub-program,  each  method  becomes  a subject  of 
analysis.  In  the  following,  we  illustrate  object  clustering  with  an  example. 

To  demonstrate  the  object  partitioning,  consider  the  dining  philosophers’  prob- 
lem. One  way  to  implement  this  problem  is  to  define  a philosopher  and  a chopstick 
as  a class.  Each  instance  of  the  philosopher  class  Philosopher-i  for  1 < i < 5,  is 
an  active  object  and  each  instance  of  the  chopslick  class,  Stick-i  for  1 < i < 5,  is 
a passive  object.  The  behavior  of  the  active  object  Philosopher-i  can  be  specified 
as  ‘SEQ(think,acquirc-stick(i),acquire-stick(i+l  mod  5),eat,  rclease-stick(i),release- 
stick(i-t-l))’  in  which  ‘think’  and  'eat'  are  methods  in  the  Philosopher,  ‘acquire- 
stick(i)'  and  ’release-stick(i)’  are  methods  in  Stick-i  and  ’acquire-stick(i+l)’  and 
‘release-stick(i+l  mod  5)'  are  methods  in  Stick-i+1.  The  meaning  of  each  method 
is  self-explanatory.  Since  the  method  invocation  defined  in  the  same  object  is  not 
necessary  to  specify,  the  problem  can  be  modeled  as  shown  in  Figure  5.2  in  which 
each  Philosopher  is  labeled  as  Pi  and  each  Stick  is  labeled  as  S,. 
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Figure  5.2.  An  object  clustering  in  the  five  dining  philosopher's  problem 
When  we  apply  the  clustering  approach  we  presented  in  the  previous  subsection, 
a possible  solution  is  shown  as  dotted  lines. 


As  we  have  discussed  above,  the  parallelism  must  be  managed  in  such  a way 
that  the  communication  overhead  can  be  controlled.  The  IPR  can  be  used  as  a task 
precedence  graph  for  partitioning  analysis. 

In  the  method  partitioning,  the  proper  grain  sizes  are  determined  by  execution 
and  communication  time  analysis.  In  the  case  of  cyclic  expression,  such  as  looping 
and  recursions,  we  consider  only  one  pass  of  such  cyclic  expression.  The  idea  of  con- 
sidering only  one  pass  is  to  determine  the  proper  grain  sizes  so  that  the  completion 
time  can  be  reduced.  This  one-pass  approach  can  be  used  to  determine  the  optimal 
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number  of  processors.  Note  that  the  optimal  number  of  processors  remains  the  same 
regardless  of  the  number  of  times  the  expression  is  repeated.  We  consider  this  step  the 
core  for  exploiting  small  grain  parallelism  in  comparison  to  the  object-partitioning 
step  in  which  large  grain  parallelism  among  objects  is  exploited.  In  order  to  perform 
the  grain  size  analysis  based  on  the  tradeoff  between  parallel  execution  and  commu- 
nication overhead,  we  estimate  the  execution  time  of  each  node  in  the  IPR  and  the 
communication  time  between  the  two  adjacent  nodes  by  analysing  the  assembly  code 
corresponding  to  the  target  code  for  each  IPR  construct.  In  the  following  sections, 
we  present  the  existing  grain  size  analysis  techniques,  and  our  grain  size  determi- 
nation approaches  on  three  different  types  of  parallelism:  pipelined  parallelism,  tree 
parallelism  and  graph  parallelism. 

IL2J Gain.  Size.  Analysis 

In  this  section,  we  briefly  compare  two  different  strategics  to  determination  of 
grain  sizes  and  present  our  methods  on  various  types  of  parallelism. 

The  existing  grain  size  determination  strategies  can  be  divided  into  two  categories 
based  on  how  the  grain  size  is  determined:  programmer  control  and  automatic  deter- 
mination. In  the  programmer-controlled  approach,  programmers  are  fully  responsible 
for  determining  the  grain  sizes,  as  well  as  expressing  explicitly  parallelism.  The  pro- 
grammers can  use  parallel  language  constructs  indicating  the  tasks  to  be  executed 
in  parallel.  When  a programmer  has  specific  information  about  the  behavior  of  a 
program,  the  programmer  can  determine  the  sizes  of  tasks.  When  that  program  is 
ported  to  a different  parallel  computer,  the  sizes  of  tasks  need  to  be  changed  to  best 
fit  to  the  new  processors.  In  addition,  it  may  not  be  easy  for  the  programmers  to 
make  decisions  on  the  sizes  of  the  tasks  due  to  lack  of  information.  This  approach 


is  closely  related  to  the  second  category  of  parallel  programming  approach  in  which 
parallel  language  constructs  are  used  for  expressing  parallelism  and  communication. 

On  the  other  hand,  in  the  automatic-determination  approach,  grain  sizes  are 
determined  automatically.  This  approach  can  be  further  divided  into  two  classes: 
the  compiler  approach  and  run-time  approach.  In  the  automatic  determination  at 
compile  time  approach,  the  programmers  do  not  provide  any  information  regarding 
the  granularity.  During  compilation,  heuristics  are  used  to  statically  determine  the 
sizes  of  tasks  (58,  59,  70J.  One  disadvantage  is  that  some  information  may  not  be 
available  before  the  run-time. 

In  the  run-time  approach,  only  simple  heuristics  can  be  applied  to  determine  the 
sizes  due  to  the  costly  overhead.  For  example,  as  in  [34],  each  recursive  function 
call  creates  a new  task  to  be  assigned  to  a processor.  In  this  case,  since  functional 
programming  involves  frequent  recursive  function  calls,  it  is  likely  that  too  many 
small  tasks  will  saturate  the  system.  In  general,  such  run-time  approaches  ignore 
the  size  of  tasks  under  the  assumption  that  there  are  reasonably  many  processors 
available.  The  automatic  determination  approach  looks  more  promising  since  the 
programmers  need  not  worry  about  the  grain  size  at  all.  However,  some  parallelism 
may  not  be  detected  without  help  front  programmers. 

In  our  approach,  we  integrate  the  two  strategies  by  utilizing  information  available 
at  the  compilation  lime  and  also  allowing  programmers'  control.  The  grain  size  is 
determined  based  on  the  analysis  of  the  execution  and  communication  times  which 
can  be  obtained  during  the  compilation  time.  The  programmers  can  analyze  the  IPR 
and  select  proper  strategics  depending  on  the  characteristics  of  the  parallelism. 

We  make  the  following  assumptions  about  the  underlying  MIMD  parallel  com- 
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• The  computer  system  consists  of  fully  connected  identical  processors  having  the 
same  processing  capability. 

• Each  processor  h 
multaneously. 

• The  communication  cost  between  two  processors  depends  only  on  the  data 
size  to  be  transmitted.  Currently,  we  ignore  the  time  required  to  set  up  the 


a capability  of  performing  program  execution  and  I/O  si- 


• Communication  cost  between  two  tasks  residing  on  the  same  processor  is  small 
enough  to  ignore. 


We  define  the  notations  wi 


in  this  section  as  the  following: 


Definition  S.S.t  An  execution  time  for  a node  n,  denoted  as  e(n),  is  a time  required 
to  complete  the  computation  represented  by  n without  being  interrupted. 

Definition  5. ft. 8 A communication  time  between  two  nodes m and n,  denoted  asc(m,n) 
is  a time  required  to  transmit  data  from  ■ to  n under  the  assumption  that  b and  n 
are  assigned  to  the  adjacent  processors. 

Definition  5.3.3  A completion  time  for  a code  c with  n number  of  processors,  de- 
noted as  E(c,n),  is  a time  required  to  finish  all  the  computation  and  communication 
involved  in  the  code  c. 

We  illustrate  the  importance  of  grain  size  in  parallel  processing  with  an  example. 
Suppose  that  we  have  a PROOF/L  code,  called  HI,  such  as  a(  b (d(vi),  e(uj)), 
c (X(us),  g(e4))).  The  IPR  for  Ml  is  shown  in  Figure  5.3.  The  graph  consists  of 
seven  nodes,  a,  b,  c,  d,  e,  f,  g,  and  six  edges,  which  represent  data  precedence 
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PI  P2  P3  P4 

Figure  5.3.  An  IPR  representation  for  Ml 

relations.  This  tree-type  parallelism  is  a typical  form  resulting  from  a divide-and- 
conquer  algorithm. 

Assuming  that  there  arc  enough  processors,  a simple  approach  would  be  to  assign 
four  tasks  d,  a,  f and  g to  four  different  processors  Pi , P2,  P3  and  P4,  respectively. 
After  completing  the  execution  of  these  four  tasks.  Pi  and  P3  continue  the  execution 
of  the  nodes  b and  c,  respectively.  Then,  Pi  executes  a to  complete  the  execution. 

In  this  case  E(M1,4)  can  be  calculated  as  follows: 

E(M1,4)  = 5 + 10  + 2 + 10  + 1 = 28 

This  result  is  not  desirable,  since  if  we  only  utilizeonc  processor,  E(M1 , 1)  • e(i)  = 
5 x 44-2  x 2+1  = 25.  Thus  E(H1,1)  < E(H1, 4),  and  there  is  no  gain  in  parallel  pro- 
cessing because  the  communication  overhead  has  overshadowed  the  gain  of  parallel 
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■S.3.2  Pipd‘"-<l  Parallelism 

One  of  the  patterns  of  parallelism  is  pipelined  parallelism.  In  exploiting  pipelined 
parallelism,  one  important  consideration  is  the  number  of  the  segments,  that  is,  how 
to  divide  the  entire  process  into  a set  of  segments  to  reduce  the  completion  time.  In 
the  following,  we  show  that  we  can  find  the  optimal  size  of  segments,  i.e.  the  proper 
grain  sizes. 

n.fmliVm  S.S.i  A dominant  segment  is  a sub-process  in  a pipelined  process  such  that 
the  completion  time  of  the  entire  process  is  dominated  by  that  sub-process. 
p,faiii, mi  S.S.5  All  the  sub-processes  other  than  the  dominant  segment  are  called 
subordinate  segments. 

The  dominant  segment  dictates  the  completion  time  of  the  entire  pipelined  pro- 
cess. In  other  words,  the  dominant  segment  is  always  busy  once  all  the  segments  are 
filled  with  data. 

X..1.6  A computation  segment  is  a sub-process  whose  task  is  to  receive 
input  data,  execute  the  computation  with  it  and  return  the  result. 

p.foitii in  .5  .9. 7 A communication  segment  is  a sub-process  whose  task  is  to  pass  data 
from  the  preceding  computation  segment  and  to  the  succeeding  computation  segment. 

p.fn.'f.'iin  s.3.8  A one-pass  completion  time,  E(a,)  defined  for  a segment  a,-,  is  an 
amount  of  time  required  to  complete  processing  of  a datum  in  a,-. 

Note  that  the  dominant  segment  can  be  either  a computation  segment  or  a com- 
munication segment.  Computation  segments  and  communication  segments  have  dif- 
ferent properties.  Computation  segments  can  be  divided  to  reduce  the  one-pass  lime. 
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However,  communication  segments  cannot  be  so  refined.  This  distinction  implies  that 
in  order  to  find  the  optimal  grain  sizes  we  need  to  begin  the  analysis  from  the  smallest 
grain  size  available.  In  our  approach,  the  IPR  presents  the  smallest  grain  parallelism 
we  can  get  in  the  program  statement  level. 

We  make  the  following  assumptions  before  we  introduce  an  optimal  solution  for 
the  determination  of  the  proper  grain  sizes. 

• The  one-pass  completion  time  for  each  segment  is  fixed  and  known  a priori. 

• There  are  a sufficiently  large  number  of  data  elements. 

• The  execution  of  code  and  communication  can  be  done  simultaneously. 

• The  communication  link  can  deliver  only  one  result  at  a time. 

Lemma  5.8.1  There  is  always  a dominant  segment  in  a pipelined  process. 

Proof  of  Lemma  5.8. 1 Under  the  assumption  that  each  segment  requires  a fixed  amount 
of  time  for  completing  its  processing  of  a datum,  the  proof  is  triviaL 

The  goal  of  finding  proper  grain  sizes  in  a pipelined  process  is  to  reduce  the  one- 
pass  completion  time  of  the  dominant  segment  either  combining  a dominant  segment 
and  its  neighboring  segments  or  refining  a dominant  segment.  However,  since  the 
IPR  presents  as  fine  parallelism  as  possible,  the  refinement  of  the  dominant  segment 
would  not  be  considered. 

Algorithm  5.8.1  Determine  Grains  for  Pipe-lined  Parallelism 
input:  An  IPR  representing  pipelined  parallelism 
output:  Optimal  grain  sizes 
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Step  1 Sort  the  segments  using  one-pass  completion  lime  as  a keg 
in  descending  order. 

Step  2 Find  a dominant  segment.  Let  the  dominant  segment  be  Si  and 

its  preceding  and  succeeding  segments  be  Si- \ and  S;+i,  respectively. 
Step  3 If  S,  is  a communication  segment,  then 
If  B(Si-,J  + E(Si+l)  < E(Si)  then 

Delete  Si-i,  Si  and  Sltl  from  the  list 
Add  new  S,‘  into  the  list  such  that 

E(S/)  = B(Si-,)  + E(Sttl) 

Go  to  Step  2 

Stop 


In  the  following  it  is  shown  that  the  algorithm  5.3.1  yields  an  optimal  solution. 
Theorem  5.3.1  The  algorithm  5.3.1  always  finds  optimal  grain  sises. 

Proof  of  Theorem  5.3.1  By  Lemma  5.3.1,  the  Step  2 will  always  find  a dominant 
segment.  When  the  dominant  segment  is  a computation  segment , since  we  cannot 
reduce  the  computation  any  further,  the  grains  cannot  be  further  clustered.  Thus,  the 
dominant  segment  remains  the  same,  and  so  does  the  completion  lime  of  the  entire 
process.  Now  consider  a case  in  which  the  dominant  segment  is  a communication 
segment.  In  this  case  the  algorithm  tries  to  partition  the  dominant  segment  with  its 
neighbor  computation  segments  to  reduce  the  one-lime  pass  time.  Let  the  sequence 
of  the  partitioning  obtained  by  the  algorithm  5.3.1  be  called  PI.  dssnme  that  then  is 
another  sequence  of  the  partitioning  called  P2  which  eon  lead  to  a belter  solution  than 
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PI.  In  P2,  a communication  segment  S,  is  first  partitioned  with  its  neighbors 
and  Sj.,  before  S,  when 

£(S.)  > E(Sj).  fl) 

Partitioning  of  Sj  with  its  neighbors  Sj+,  and  5j_i  means  that 

E(S,.,)  + E(Sltl)  < E(S,)  ft) 

prom  (I)  and  (2)  we  know  that  EfSj. ,)  + £(S,+i)  < £(S.) 

U Si  and  Sj  art  two  adjacent  communication  segments,  then  in  P2  it  may  not  be 
possible  to  partition  Si  with  Si.,  and  S,tl  (=  Sj.,)  because  E(Sitt)  has  increased  to 
E(Siti)  -f  E(Sjti)-  Thus,  the  dominant  segment  Sis,  may  remain  unpartitioned  in 
P2.  Therefore,  P2  cannot  yield  an  optimal  solution.  We  proved  by  contradiction  that 
the  algorithm  5.S.I  can  always  yield  an  optimal  solution. 

Let  the  number  of  segments  be  n.  Then,  the  algorithm  5.3.1  can  find  an  optimal 
grain  sizes  during  compilation  in  the  time  complexity  shown  in  the  following  theorem. 
Theorem  5.3.2  The  time  complexity  of  the  algorithm  5.S.I  is  Ofnlogn). 

Proof  of  Theorem  5.3.2  Step  I requires  nlogn  steps.  By  using  a priority  queue  data 
structure  to  store  the  sorted  list,  we  can  retrieve  and  store  any  element  in  logn  step. 
Thus,  each  execution  of  Step  S requires  at  most  4 logn  steps  for  three  deletions  and 
one  addition  to  the  priority  queue.  Since  there  are  at  most  n segments,  the  entire 
algorithm  can  run  Ofnlogn). 

5.3.3  Tree  Parallelism 

We  consider  a case  in  which  the  task  precedence  graph  is  a tree.  Before  we  present 
our  approach,  we  define  the  terms  we  use. 

Urfmilion  $,}.}  The  one-level  sub-tree  is  a subset  of  nodes  uo,ti|,...,e„  in  a tree 
such  that  vo  is  a parent  node  of  all  the  nodes  v,,Vj,...,vn. 
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In  the  following,  we  call  the  one-level  sub-tree  simply  sub-tree.  The  number  of 
sub-trees  in  a tree  is  the  same  as  the  number  of  non-leaf  nodes. 

Definition  5.3.10  A task  precedence  tree  7p  is  a tree  in  which  each  node  represents  a 
computation  and  each  edge  specifies  the  data  dependency  relationships  among  nodes. 

Parallelism  obtained  from  divide-conqucr  strategy  can  lead  to  the  parallelism  of 
tree  pattern  and  thus  be  represented  by  Tp. 

Definition  5.S.1I  A gain-tree  Tp  of  Tp  is  a weighted  tree  in  which  each  node , called 
a gain  node,  represents  a sub-tree  in  Tp  and  each  edge  represents  data  dependency 
relations  among  the  nodes.  Each  gain  node  has  a weight,  called  gain,  corresponding 
to  an  amount  of  possible  maximum  contribution  to  reducing  the  completion  time  when 
the  corresponding  sub-tree  is  clustered. 

A gain  for  a sub-tree  consisting  of  ni , . , . , nm  is  denoted  as  GAI N[nt. . . . , nm).  Our 
grain  size  determination  approach  can  be  considered  as  a horizontal  clustering  or 
partitioning  in  that  a set  of  adjacent  nodes,  i.e.  asub-tree,  is  considered  as  a candidate 
for  clustering.  The  essential  part  of  our  grain  size  determination  approach  is  to 
estimate  the  possible  contributions  which  can  be  made  by  clustering  the  adjacent 
nodes.  Our  approach  consists  of  two  parts: 

1)  build  a gain-tree  from  a given  input  task  precedence  graph,  and 

2)  determine  grain  sizes  from  the  gain-tree. 

The  gain-tree  can  be  built  by  analyzing  each  sub-tree  in  the  task  precedence  tree 
using  the  following  procedure: 

Procedure  Gain-Analysis 

input:  a sub-tree,  consisting  of  s and  its  children  ni, . . . ,nm 
output:  GA/N(s, ni, ...  ,nm) 
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Step  1 Calculate  the  total  execution  time  tc  in  one  processor 
tc  = «(»)  + £!"=.  «(".)• 

Step  2 Find  a node  m,  for  1 < k < m,  such  that 

pc  = c(nk)  + c(nit,s)  is  the  second  largest 

Step  3 If  pc  + c(s)  > tc  then 

GAIN[a,  n„  n2 nm)  = pc+  e[s)  - tc 

GAlN(a,num n„)  = 0 

In  Step  1,  the  bottleneck  time  used  to  decide  whether  the  computation  represented 
by  a sub-tree  is  or  is  not  executed  in  parallel  is  determined  by  summing  all  the 
execution  times  of  the  nodes  in  the  sub-tree.  In  Step  2,  the  time  required  to  complete 
the  computation  represented  by  the  sub-tree  when  the  nodes  in  that  sub-tree  are  not 
clustered  is  calculated.  Because  at  the  scheduling  stage  one  of  the  child  nodes  can 
be  scheduled  to  the  same  processor  as  the  node  s and  by  the  static  analysis  the 
scheduler  can  choose  a node  n,  such  that  e(nj)  + c(n„s)  is  the  largest,  the  second 
largest  schedule  length  is  calculated  to  be  used  as  an  actual  time  required  to  complete 
the  computation  represented  by  s,ni,...,nm.  In  Step  3,  a gain  is  calculated  by 
comparing  the  bottleneck  time  with  the  actual  completion  time  calculated  in  Step  2. 
If  there  is  a positive  gain,  then  the  amount  of  the  gain  calculated  is  assigned  to  the 
gain  node.  Otherwise  the  gain  is  set  to  zero. 

The  time  complexity  of  the  procedure  Gain-Analysis  can  be  determined  by  the 
following  analysis.  Step  1 requires  0(1)  time,  Step  2 requires  0(m)  time  in  which 
m is  a number  of  child  nodes  and  Step  3 requires  0(1)  lime.  Thus,  the  exact  time 
complexity  of  the  procedure  Gain-Analysis  is  0(m). 


109 


Now,  we  build  a pain-free  T,  from  a precedence  tree  Tr  using  the  procedure  Gain- 
Analysis  in  the  following  manner: 

Algorithm  5.3.2  Build  Gain  Tree 
input:  a task  precedence  tree  Tr 
output:  a join-tree  Tt 
For  all  sub-trees  t.  do 

Let  t,  consist  of  a node  s and  its  children  ni,na,. . . ,n„ 

Step  1 Call  Gain-Analysis^s,  ni,  nr, , nm) 

Step  2 Connect  the  gain  node  to  the  existing  gain  nodes 

In  Step  1,  Gain- Analysis  is  called  for  each  sub-tree  to  determine  the  possible  contri- 
bution to  the  reduction  of  the  completion  time  when  the  sub-tree  is  clustered.  In  Step 
2,  the  newly-created  gain  node  for  the  current  sub-tree  is  connected  to  the  existing 
gain  nodes.  The  construction  of  the  gain-tree  can  be  done  by  omitting  leaf  nodes 
from  the  original  task  precedence  tree  and  associating  the  gain  calculated  in  Step 
1 to  a parent  of  the  corresponding  sub-tree.  Step  1 requires  O(m)  where  m is  the 
number  of  the  child  nodes.  Since  Step  1 is  executed  for  each  sub-tree  and  the  number 
of  sub-trees  is  bounded  by  the  number  of  non-leaf  nodes  in  the  tree,  the  algorithm 
needs  to  visit  each  node  once.  In  Step  2,  each  node  also  needs  to  be  visited  once. 
Thus,  the  time  complexity  of  the  algorithm  5.3.2  is  O(n)  where  n is  the  number  of 
nodes  in  the  tree. 

Once  the  gain-tree  is  built,  the  grain  size  can  be  determined  by  selecting  dusters 
heurislically.  Our  grain  size  determination  is  based  on  the  observation  that  contribu- 
tion  from  the  nodes  dose  to  the  root  node1  propagates  to  the  other  nodes.  In  order 


: is  a node  having  depth  of  0 
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(a)  (b) 

Figure  5.4.  Simple  gain-tree  examples 

to  illustrate  this,  suppose  that  we  have  simple  gain-trees  as  shown  in  Figure  5.4  in 
which  V(,  1 < i < 3,  represent  a set  of  gain  nodes  and  o,  6,  c represent  an  amount 
of  the  gain  for  each  node.  In  Figure  5.4  (a),  the  precedence  relations  are  Vj  — * t)| 
and  Vi  -•  t»i.  In  Figure  5.4  (b),  the  precedence  relations  are  i>i  -»  uj  and  »|  -•  03. 
The  goal  of  the  analysis  is  to  select  the  best  two  candidates  for  clustering  so  as  to 
increase  the  overall  contribution  to  the  reduction  of  the  completion  time.  The  overall 
contribution  is  determined  by  the  following  equation: 
overall  contribution 

= nin(  overall  contribution  from  a path  between  ui  and  02, 
overall  contribution  from  a path  between  ui  and  uj  ) 

Thus,  when  nin(5,c)  < a,  v,  and  a node  having  a weight  of  >ax(6,c)  are  chosen  to 
cluster,  and  when  min(i,c)  > 0,  v}  and  03  are  chosen. 

Note  that  the  rules  presented  above  can  be  applied  to  both  gain-trees  shown  in 
Figure  5.4.  It  implies  that  our  gain  size  analysis  technique  can  be  used  for  the  analysis 


Ill 


of  both  in-tree  form  as  in  Figure  5.4  (a)  and  out-tree  form  a a in  Figure  5.4  (b).  The 
following  is  an  algorithm  to  determine  grain  sizes: 


input;  A gain-tire.  Ts 

output;  A clustered  task  precedence  tree 


Step  1 Initialise  all  the  nodes  as  ‘unclustered’ 

Step  2 Sort  the  nodes  in  Tg  using  the  gain  as  a primary  key  in  descending 

order  and  the  depth  oj  each  node  as  a secondary  key  in  ascending  order 
Step  3 Get  the  largest  node  ni  in  the  sorted  list 

Step  3.1  Ifni  is  not  a root  node  and  a parent  of  ni  and 
any  sibling  of  n;  are  ‘ clustered ' then 
go  to  Step  9-f 

Step  3.2  If  two  or  more  child  nodes  of  n i are  set  ‘clustered’  then 
go  to  Step  9-4 
Step  3.3  Set  n,  as  ‘clustered’ 

Step  3.4  Delete  m from  the  list 
Step  3.5  If  there  is  a node  with  a positive  gain  then 
go  to  Step  9 


gains  of  the  possible  candidates.  Step  1 requires  O(n)  time  to  visit  each  node  once. 
In  Step  2,  0(nlogn ) time  is  required  to  sort  n number  of  nodes.  Step  3 must  visit 


112 


adjacent  nodes  at  most  once  for  each  node.  Because  the  number  of  the  adjacent 
nodes  is  same  as  the  number  of  edges  in  a node  and  each  edge  will  be  visited  at 
most  twice,  the  overall  time  complexity  in  Step  3 is  bound  to  0(e)  in  which  e is 
the  number  of  the  edges  in  T,.  Therefore  the  time  complexity  of  this  algorithm  is 
0(max(ntogn,  e)).  Note  that  in  the  case  of  trees  the  complexity  yields  to  O(nlogn). 
Since  this  algorithm  is  also  used  in  the  case  of  the  graphs  later,  we  define  the  time 
complexity  of  this  algorithm  as  above. 

In  the  following,  we  compare  our  approach  to  McCreary’s  approach  [58).  Mc- 
Creary uses  an  algorithm  of  the  complexity  0(n3)  that  decomposes  a graph  into  a 
set  of  clans  that  are  classified  as  primitive,  linear,  or  independent.  When  the  clans 
are  labeled  as  independent,  the  possibility  of  parallelization  exists.  However,  when 
the  clans  are  labeled  as  primitive  or  linear,  they  are  grouped  together  and  executed 
sequentially.  In  order  to  compare  our  approach  to  McCreary’s,  we  use  the  same  ex- 
ample used  in  (58).  The  task  precedence  tree  of  the  example  is  shown  in  Figure  5.5. 
Every  node  has  a unique  number  within  the  circle  representing  that  node,  and  a 
weight  is  attached  to  the  node.  The  communication  cost  is  assigned  to  each  edge. 
For  instance,  a node  9 has  a weight  1 and  each  of  the  three  edges  has  communication 
cost  18.  Using  McCreary's  approach,  the  schedule  and  its  completion  time  are  shown 
in  Figure  5.6. 

The  problem  with  McCreary's  approach  is  that  it  begins  to  search  the  candidates 
for  parallel  execution  from  the  bottom  of  the  tree.  Once  the  clustering  is  done  at  the 
lower  part  of  the  tree,  the  clustering  at  the  higher  level  is  less  likely  to  occur,  since  such 
clustering  may  have  to  sacrifice  the  possibility  of  parallel  execution  at  the  lower  level. 
In  addition,  the  key  concept  of  the  graph  decomposition  approach,  clan,  is  determined 
without  using  information  regarding  the  execution  time  and  communication  cost. 
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Figure  5.7.  A gain-tree  for  the  tree  parallelism  example 
Our  grain  size  determination  approach  uses  the  execution  and  communication 
times  to  determine  the  proper  grains  at  the  beginning.  By  analyzing  the  contribution 
of  the  clustering  locally  in  each  sub-tree,  we  build  the  gain-tree.  Then,  we  first  select 
the  largest  gain  in  the  gain-tree  as  the  candidate  for  the  clustering  and  continue  the 
selection  until  all  the  positive  gains  are  processed.  The  gain-tree  for  this  example 
is  shown  in  Figure  5.7  in  which  a calculated  gain  is  shown  within  each  gain  node. 
The  corresponding  nodes  in  the  task  precedence  tree  shown  in  Figure  5.5  arc  also 
associated  with  each  gain  node. 

From  the  information  obtained  in  the  gain-tree,  we  can  determine  five  grains  as 
follows: 

Cl  = { 1,  2, 9, 10, 13,  14, 15  } 

C2  = { 5,  6, 11  } 
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Figure  5.8.  A schedule  obtained  from  our  approach 
C3  = { 7,  8,  12  } 

C4  = {3} 

CS  = {4} 


Using  these  clusters,  we  show  two  schedules  and  their  completion  time,  in  Fig- 
ure  5.8,  A schedule  for  four  processors  is  shown  (a)  i„  Figure  5.8  and  „ achedu,c  fw 
five  processors  is  shown  (b)  in  Figure  5.8. 


In  the  following,  we  present  a task  allocation  strategy  which  can  produce  an 
optimal  solution  in  a restricted  case.  We  first  present  the  algorithm  and  then  show 
that  it  can  find  an  optimal  solution  when  the  following  assumptions  are  met. 

• The  task  precedence  graph  is  a tree. 

• There  are  sufficiently  many  processors  so  that  the  leaf  nodes  in  the  task  prece- 
dence graph  can  be  executed  simultaneously,  if  necessary. 

• The  grain  size  analysis  has  been  completed  and  no  more  clustering  of  grains  to 
reduce  communication  time  is  necessary. 

This  algorithm  can  be  considered  as  a variant  of  list  scheduling.  The  priority  of  the 
task  for  allocation  to  the  same  processor  is  decided  by  the  urgentness  of  the  task. 
This  algorithm  also  uses  the  concept  of  the  gain  analysis.  The  algorithm  tries  to 
reduce  the  completion  time  of  the  each  node  by  clustering  the  critical  path  with  that 

Algorithm  5.3. i Task  allocation  for  tree 

input;  A task  precedence  tree 
output;  a set  oj  clusters 

Step  1 Identify  a node  s whose  predecessors  are  all  leaf  nodes,  n,-  for  1 < i < m. 

If  there  is  no  such  a node  s,  then  stop. 

Step  2 If  s has  only  one  predecessor,  n,  then  replace  s,  n and  an  edge 

between  s and  n with  a new  node  n such  that  e(n')  = e(s)  + e(n)  and 
go  to  step  I. 

For  s and  n„  I <i<m,  do  the  following: 


Step  3 
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Step  3.1  Calculate  the  total  execution  lime  in  one  processor 

<e  = e(s)  + Er=.«W- 

Step  3.2  Find  a node  np,  for  1 < p < m,  such  that 
e(np)  + c(np,  s)  is  the  largest. 

Step  3.3  Find  a node  n,,  for  1 <q<m,  and  p ^ q,  such  that 
ca  = e(n,)  + c(nflt  s)  is  the  largest. 

Step  3.4  If  Ci  > tc-  e(s)  then 

cluster  s and  n,  into  a new  node  y such  that 
y = a U ni  U nj . . . U n,  and  e(y)  = tc 
delete  s and  all  n,'  from  the  tree 
attach  y in  the  place  of  a 


and  represent  it  as  a 


such  that  r = a U np  and 

e(r)  = max(e(np)  + e(a),e(n,)  + c(n 
delete  a and  all  n,  from  the  tree 
attach  r in  the  place  of  a 


Step  3.5  Go  to  step  I. 


We  show  in  the  following  theorem  that  Algorithm  5.3.4  produces  an  optimal 
allocation  in  the  restricted  case. 

Theorem  5.3.3  Algorithm  5.S.f  produces  an  optimal  allocation  when  the  following 
assumptions  are  satisfied: 

I)  The  task  precedence  graph  is  a tree.  2)  Then  an  a greater  number  of  processors 
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than  the  number  of  leaf  nodes  in  the  task  precedence  graph.  3)  No  more  clustering  is 

Proof  of  Theorem  5.3.3  If  assumption  3)  is  satisfied,  C;  is  always  less  than  or  equal 
to  tc  — e(s)  in  step  S.f  of  Algorithm  5.3. f.  Thus,  the  else  part  of  step  S.f  will 

s (the  time  required  to  complete  the  node  s and  its  descendants J is  only  dependent 
on  how  the  node  s and  its  descendants  are  allocated.  Thus,  we  only  need  to  show 

precedence  graph.  Suppose  that  there  is  a subgraph  consisting  of  a node  s and  its 
predecessors  n,*,  for  1 < i < m.  Let  the  allocation  produced  by  Algorithm  5.3.f  be  A, 
and  the  time  required  to  complete  the  tasks  in  this  subgraph  using  A be  C/,.  Suppose 
that  we  have  another  allocation,  called  B,  for  this  subgraph,  and  the  time  required 
to  complete  it  in  B is  Cb,  and  Cb  < CA.  In  the  following,  we  show  that  Cb  < Ca 
cannot  be  true.  In  A,  all  n,-  except  np  wilt  be  allocated  to  different  processors,  and 
in  each  cycle  of  Algorithm  5.3.  f only  one  task  is  chosen  to  cluster  with  its  successor, 
and  the  addition  of  any  other  tasks  to  this  cluster  will  not  reduce  the  completion  time 
by  the  assumption  3).  In  B a different  task,  nf<,  other  than  np,  would  be  selected  in 
step  3.2  to  cluster  with  its  successor  s.  In  step  3.3,  np  will  be  chosen  instead  of  n„ 
because  e(np)  + c(np,s)  is  the  largest. 

Then,  CB  = max(e(nr)  + c(np,s)  + e(s),e( np.)  + e(s)). 

Since  we  know  that  e(np)  + c(np,s)  > c(nj), 

CB  = «(n,)  + c(n„s)  + C(s).  (!) 

In  A,  CA  = max(e[np)  + e(s),e(n,)  + e(n„s)  + e(s))  (2) 

Therefore,  there  are  two  cases. 

case  I)  if  e(np)  + e(s)  > e(n,)  + c(n„s)  + e(s)  then  in  (2) 


(3) 


Cm  = «(  »,)  + «(«) 

from  (1)  and  (S),  wo  know  that  C„>CAisa  contradiction, 
case  2)  if  c(n,)  + <**„»)  + «(3)  > e(n,)  + e{s)  then  in  (2) 

from  (1)  and  (4),  to  satisfy  CB<CA,  the  following  agnation  should  be  true: 


«=("„)  + «K.*)  + C[s)  < «K)  + c(n„s)  + e{s) 


complexity  o/0(r>)  when  n 
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Definition  5.3.  IS  A depth  of  a node  in  a graph  is  the  length  oj  the  longest  path  from 
the  highest  ancestor  of  that  node. 

A node  having  no  incoming  edge  is  of  depth  0. 

Definition  5.3.  IS  A height  of  a graph  is  the  largest  depth  of  the  graph. 

A gain-graph  Gt  and  a task  precedence  graph  Gp  are  defined  in  the  same  manner 
as  the  gain-tree  and  task  precedence  tree  except  that  they  arc  graphs.  The  nodes  in 

precedence  graph.  In  the  graph  cases,  we  also  analyze  gains  to  determine  the  proper 
grain  sizes.  We  first  build  a gain-graph  and  apply  the  algorithm  5.3.3  to  determine 
grain  sizes.  The  following  is  an  algorithm  to  build  the  gain-graph: 

Algorithm  5.3.5  Build  Gain  Graph 

input:  A task  precedence  graph  G,  = (V,  E ),  | V |=  n and  | E |=  e 
output;  A gain-graph  in  which  each  node  represents  a sub-tree  in  the  graph 
Step  1 Determine  the  depth  of  each  node  in  Gp 
Step  2 Set  all  the  nodes  ‘ unmarked ' 

Step  3 For  a depth  d = 0 to  a height  k do 

Step  3.1  Find  predecessors,  n ] , n 2 , . . . , n ,n  of  s having  depth  d — 1 
if  they  are  not  all  ‘marked’  then 

Call  Cain-Analysis(s,ni,nj, . . .,nm) 

if  the  out-degree  of  s is  greater  than  I then 
set  s ‘ marked ' 


Connect  the  gain  node  with  the  existing  gain  nodes. 


122 


Step  3.2FiW  the  successors  >11,119, ...  ,nm  °1 3 having  depth  i + 1 
Call  Gain-Analysis(s,ni,nj, . . . , nm) 

Set  n i,ri9,...,n,n  'marked' 

Connect  the  gain  node  with  the  existing  gain  nodes. 

In  Step  1,  the  depth  of  each  node  can  be  determined  using  a breadth  first  search 
method.  Each  node  may  be  involved  in  more  than  one  path,  and  consequently  can 
have  a different  length  for  each  path.  In  such  cases,  by  the  definition  of  depth , the 
largest  value  is  set  to  the  depth  of  that  node.  In  Step  2,  all  the  nodes  are  initially 
set  as  'unmarked',  meaning  that  the  node  is  not  involved  in  any  clustering.  In  Step 
3,  the  gain-graph  is  built  by  visiting  each  node.  First,  a node  having  smallest  depth 
is  chosen,  and  its  predecessors  are  first  checked  to  determine  whether  all  of  them 
are  already  included  in  any  cluster.  If  they  were  not  included  in  any  cluster,  then 
GAIN  is  calculated.  The  predecessors  are  all  set  ‘marked’,  and  the  node  is  set 

consideration  is  created.  Edges  are  established  from  the  existing  gain  nodes  to  the 
newly-created  gain  node  if  there  is  a node  common  to  both  the  new  gain  node  and 
the  existing  nodes.  The  same  steps  are  applied  to  the  successor  nodes  except  that  all 
the  nodes  set  'marked’. 

The  time  complexity  of  this  algorithm  is  analyzed  as  follows: 

Steps  1 and  2 require  visiting  each  node  once,  and  thus  O(n)  time  is  required.  The 
complexity  of  Step  3 is  dominated  by  the  procedure  Gain-Analysis,  and  thus  each 
pass  of  Step  3 requires  0{m  + m ')  where  m and  m'  are  the  number  of  predecessors 
and  the  number  of  successors,  respectively.  Step  3 needs  to  be  executed  for  each 
node  in  the  graph.  Because  m + m'  is  the  total  number  of  edges  in  a node,  each  edge 
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needs  to  be  visited  at  most  twice  and  thus  the  time  complexity  of  this  step  is  bound 
to  0(e).  Therefore,  the  overall  complexity  of  the  algorithm  5.3,5  is  bound  to  O(e). 

Once  the  gain-graph  is  built,  we  can  apply  the  algorithm  5.3.3  to  determine  the 
proper  grains.  Then  the  newly-obtained  task  precedence  graph  as  a result  of  applying 
the  algorithm  5.3.3  can  be  used  as  input  for  the  scheduling  stage. 

To  illustrate  our  approach  we  use  the  example  used  in  [59].  The  task  precedence 
graph  for  the  FFT  (Fast  Fourier  Transformation)  is  shown  in  Figure  5.9  using  the 
same  notation  as  the  former  example.  Using  the  algorithm  5.3.5,  the  gain-graph  is 
obtained  as  shown  in  Figure  5.10. 

We  also  show  a schedule  for  the  FFT  problem  in  the  case  of  four  processors  in 
Figure  5.11.  McCreary  [59]  extended  her  approach  by  adding  analyzing  techniques 
for  some  special  patterns  of  dependency  relations  among  primitive  nodes.  Note  that 
with  her  improved  method  the  former  task  precedence  tree  example  shown  in  Fig- 
ure 5.5  results  in  the  same  schedule  because  the  former  example  does  not  include 
primitive  nodes.  Compared  to  McCreary’s  approach,  our  approach  has  the  follow- 
ing advantages:  First,  our  approach  is  more  efficient  in  terms  of  time  complexity. 
As  shown  above,  the  time  complexity  of  our  approach  is  O(max(nlogn,c))  in  which 
nlogn  is  for  the  algorithm  5.3.3  and  e is  for  the  algorithm  5.3.5.  The  time  complexity 
of  McCreary's  approach  is  dominated  by  the  parsing  algorithm  of  0(n3).  Second,  our 
approach  is  more  general  in  the  sense  that  any  acyclic  task  graph  can  be  analyzed. 
McCreary’s  method  can  handle  only  a few  patterns  of  dependency  relations  among 
nodes.  Third,  our  approach  considers  the  execution  and  communication  times  as 
primary  factors  from  the  beginning  and  selects  the  best  candidates  from  the  entire 
task  precedence  graph  based  on  the  analysis  of  such  limes.  As  discussed  before, 
McCreary's  method,  however,  may  lose  the  clustering  opportunity  at  later  stages 
because  the  decomposition  of  the  graph  is  done  without  using  such  time  information. 
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Figure  5.9.  A task  precedence  graph  for  the  FFT  problem 
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Figure  5.10.  A gain  graph  for  the  FFT  problem 
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FigureS.il-  A Schedule  for  the  FFT  problem 


CHAPTER  6 
DISCUSSION 

In  this  dissertation,  we  have  presented  (a)  the  computation  model  PROOF  as 
a basis  for  expressing  parallelism,  (b)  the  IPR  and  the  transformation  rules  as  a 
basis  for  exploiting  parallelism  and  (c)  grain  size  determination  techniques  for  three 
patterns  of  parallelism. 

We  have  developed  a computation  model  PROOF  in  which  an  object-oriented 
paradigm  is  incorporated  into  a functional  paradigm  without  sacrificing  the  advan- 
tages of  either.  PROOF  allows  the  expression  of  parallelism  on  two  different  levels: 
object  level  and  method  level.  The  referential  transparency  and  history  sensitivity 
are  integrated  via  the  pseudo-function  It.  One  of  the  problems  in  a parallel  object- 
oriented  computation  model  is  to  smoothly  integrate  synchronization  constraints  and 
inheritance.  In  PROOF,  this  integration  is  achieved  by  separating  the  synchroniza- 
tion constraint  of  each  method  from  its  pure  operation. 

We  have  developed  an  intermediate  form  IPR  to  make  parallelism  explicit  in  the 
PROOF/L  programs.  The  IPR  is  a hybrid  graphical  representation  in  which  object- 
level  behavior  is  represented  as  a Petri-net  and  method-level  behavior  is  represented 
as  a set  of  function  nodes  and  their  data  dependency  relations.  We  have  also  devel- 
oped transformation  rules  from  the  PROOF/L  programs  to  the  IPR  and  proved  that 
the  transformation  preserves  the  correctness  of  the  PROOF/L  program  by  showing 
that  the  interleaving  semantics  of  the  PROOF/L  program  can  be  retrieved  from  the 
operational  semantics  of  the  IPR.  The  separation  of  semantics  in  the  two  different 
levels  makes  verification  of  the  program  easy.  By  introducing  the  IPR,  the  semantic 
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and  performance  issues  of  the  program  can  be  treated  separately.  The  1PR  can  serve 
as  a task  precedence  graph  for  grain  size  determination  analysis.  The  programmer’s 
responsibility  to  ensure  correct  synchronization  and  communication  is  reduced  by 
inserting  the  proper  code  during  the  transformation  phase.  Considering  the  fact  that 
coding  of  synchronization  and  communication  is  highly  error-prone,  this  automatic 
coding  reduce  errors  and  thus  benefit  the  software  development. 

We  have  presented  a two-level  allocation  approach  in  which  the  objects  are  parti- 
tioned into  a set  of  clusters  and  in  each  cluster  the  proper  grain  sizes  are  determined 
within  each  method.  We  have  developed  the  grain  size  determination  algorithms  for 
different  types  of  parallelism:  pipe-lined,  tree  and  graph.  In  the  case  of  pipe-lined 
parallelism,  we  show  that  our  algorithm  can  find  optimal  solutions.  In  the  cases 
of  tree-parallelism  and  graph-parallelism,  we  compare  our  approach  to  the  existing 
methods  and  show  that  our  approach  can  perform  better  in  terms  of  the  time  com- 
plexity. 


There  are  many  potential  directions  for  the  future  work.  One  immediate  interest  is 
to  improve  the  performance  of  the  target  code  by  developing  optimization  techniques. 
We  have  experimented  on  various  PROOF/L  programs,  including  factorial  problem, 


lem  and  air-base  defense  simulation  problem.  We  have  compared  the  performance  of 
the  Inmos  C codes  generated  by  our  transformation  system  with  the  performance  of 
the  Inmos  C codes  written  by  hand  in  terms  of  the  completion  time.  Because  the 
current  Irausformauon  system  does  not  include  optimization  techniques,  the  perfor- 
mance of  the  generated  code  is  not  as  good  as  the  code  implemented  directly  on  the 
transputer  systems.  For  example,  we  have  extensively  tested  pipelined  versions  of 
the  factorial  program.  The  speed-ups  of  the  two  factorial  programs,  the  generated 
code  and  the  directly  written  code  by  hand,  are  almost  the  same,  but  the  absolute 
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completion  time  of  the  generated  code  is  at  best  twice  bigger  than  the  completion 
time  of  the  directly  written  code.  When  the  programs  involve  extensive  manipula- 
tion of  list  data  types,  the  generated  codes  require  much  more  time  than  the  written 
codes.  We  believe  that  it  is  due  to  inefficiency  of  current  list  handling  routines  im- 
plemented in  the  back-end  transformation.  In  addition  to  the  code  optimization 
techniques,  we  need  to  develop  techniques  that  can  reduce  unnecessary  data  move- 
ment during  object  invocation.  Another  interesting  aspect  is  to  extend  current  work 
to  real-time  systems,  such  as  process  control  systems,  and  to  avionics.  In  particular, 
its  extension  to  distributed  real-time  application  can  be  of  immediate  interest.  To 
do  so,  we  will  need  to  extend  or  modify  the  computation  model  in  order  to  address 
real-time  specific  issues,  such  as  timing  constraints  and  fault-tolerance.  In  addition, 
scheduling  and  allocation  strategics  need  to  be  changed,  since  in  real-time  applica- 
tions the  goal  of  scheduling  and  allocation  differs  in  that  their  goal  is  not  to  simply 
reduce  the  completion  time  but  to  schedule  and  allocate  to  meet  the  deadline  of  each 
task.  It  will  also  be  worthwhile  to  investigate  how  we  can  improve  our  computation 
model  PROOF  to  allow  dynamic  creation  of  objects.  This  extension  will  also  require 
appropriate  adjustments  in  the  entire  programming  system,  including  adaptation  of 
dynamic  scheduling  and  allocation  approaches.  We  will  also  investigate  the  extension 
of  the  computation  model  with  the  array  constructs  so  that  arrays  of  objects  can  be 
easily  created  by  programmers. 

Current  transformation  rules  for  the  front-end  translation  are  given  based  on 
lenient  semantics,  which  is  a very  weak  version  of  non-strict  semantics.  In  order  to 
make  our  approach  more  versatile,  we  also  need  to  develop  different  transformation 
rules  for  non-strict  semantics.  One  way  to  detect  parallelism  in  non-strict  semantics 
is  to  evaluate  a function  and  its  arguments  at  the  same  time.  Then  the  effect  of 
the  non-strict  semantics  can  be  determined.  In  addition,  the  transformation  rules 
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for  exploiting  conservative  parallelism  by  lazy  semantics  need  to  be  developed.  The 
comparison  of  these  three  ways  of  exploiting  parallelism  will  also  be  of  interest. 

Current  implementation  does  not  fully  support  the  CREW  model  of  execution, 
although  the  computation  model  PROOF  supports  such  a model.  Each  object  is 
either  considered  as  a read-only  object  or  a writable  object,  but  objects  can  be  com- 
bination of  both.  We  will  investigate  a technique  to  fully  support  the  CREW  model, 
including  efficient  synchronization  constraints  for  improving  parallel  execution. 
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A2.  PROOF/L  program  examples,  their  IPRs  and  generated  target  codes' 
Bounded  Buffer  PROOF/L  program’ 

Program  Bounded-Buffer: 

class  buffer  (itemtype,  size:int) 
composition 

store:!ist(ilemtypc)  X countnnt 
method  get  (buhbuffer  — * buffer,  itemtype) 
guard  (>  buf.count  0) 
expression 

[/3[tail,  dec] [buf. store,  buf.count],  hcad(buLstore)] 
method  put  (buhbuffer,  x:itcmtype  — * buffer) 
guard  (<  buf.count  size) 
expression 

^(appcnd-rigbt(x),  inc]|buf.store,  buf.count] 
method  is-empty  (buhbuffer  — * boolean) 
expression 
= buf.count  0 

method  length  (bufibuffer  — * int) 
expression 
buf.count 

end  class 

class  producer  (itemtype) 

method  produce  ( — * itemtype) 

end  class 

class  consumer  (itemtype) 

method  consume  (itemtype  — » ) 

end  class 

Object  buf:  instance  of  buffer 

Active  Object  producer:  instance  of  producer 

Active  Object  consumer:  instance  of  consumer 

Body  of  object  producer: 

while  (TVue,  7?  | buf  ] put(buf,  producc())) 

Body  of  object  consumer: 

while  (TVue,  0 |R  ( buf  ] consume()]|get(buf)]) 
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An  1PR  for  put 


An  IPR  for  get 


) 

jisSiSr;*’''’"''""''''”"'  " 
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Print < ( "Consumer  Xd\n",j»; 
receive(Fromproc[0] , message, »v 

send(Toproc[0] ,message[buf .sto 


} _ 

Print ( ("Buf f er\n") ) : 
f f lush(stdout) ; 
for  (k*0;k<buf  .count  ;k«) 
PrintCC"  7,d  “,buf . store [k]))  ; 
Print(C'\n....«......«.\n"»; 


Xd\n" , buf . count ) ) ; 


void  _producer(Process  *p,  Channel  "chanOl , Channel  • synch, Channel 
•FromprocG) 


/•struct  buffer  lv002;*/ 
struct  buffer  lv003; 

int  lv004;  8 ‘ 


router  •/ 


message  » (MESG  •)malloc(sizeof (MESG)) ; 
synch  - synch; 

for(i*0;i<20;i*+) 

{ 

ChanOutlnt(chanOl.var) ; 

ChanlnCchanOl , buf. store,  NSIZE  • 4); 
buf. count  ■ Chanlnlnt(chanOl); 

putCftbuf,  lv004,  41v003) ;/•  Changed  the  3rd  parameter  •/ 
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if  ((ToprocCi]  a ChanAllocO)  «■  NULL) 
abortO; 

} 

Fromproc  [SOFT. CHANNELS 3 » NULL; 

Toproc [SOFT, CHANNELS]  - NULL; 

control  a ProcAlloc (Router , 0 , 2 , Fronproc .Toproc) ; 

if  (control  ■■  NULL) 

tt  ■ ProcAlloc(_main, 0,2, Fromproc, Toproc); 


/•init_traca(3) ;•/ 
ProcPar (control, tt, NULL) ; 


srg^ssP' 


lililfr;. 


:n“>0! 

s'£r" 


producer  1 producer; 
/*produce(4producer,  outOl);*/ 


void  consuaeCstruct  c) 


Factorial  PROOF/L  program 


program  fact: 

class  tl(iat) 

method  fac  ( i:int  ->  int) 
expression 

if((lambda  (x)  = 0 x),l, (lambda  (x) 


(-  x l))))i 


Body  of  object  tt  : fac  0 
An  IPR  for  fac 
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Dining  Philosopher  PROOF/L  program 


class  chopstick 


method  acquire(c: chopstick  ->  chopstick) 
expression 

method  acquire(c: chopstick  ->  chopstick) 
guard  ( « c. stick  1) 
expression 


end  class 


class  philosopher 
composition 

method  think(  p:philosopher  ->  int  ) 
expression 

delay(3) 

method  eat(  p: philosopher  ->  int  ) 
delay (3) 


Active  Object  pi  : instance  of  philosopher 
Active  Object  p2  : instance  of  philosopher 
Active  Object  p3  : instance  of  philosopher 
Active  Object  p4  : instance  of  philosopher 
Active  Object  pS  : instance  of  philosopher 

Object  cl  : instance  of  chopstick 
Object  c2  : instance  of  chopstick 
Object  c3  : instance  of  chopstick 
Object  c4  : instance  of  chopstick 
Object  c5  : instance  of  chopstick 


Body  of  object  p2: 

TT73LtlJq^^ 

Body  of  object  p3: 

Body  of  object  p4: 


: object  pi. 


Active  Object  pi  : philosopher 


File  1 macro 


Node  Node  Number 

Number  Name  of  Input 

55  DIST  2 

29  TRUE  1 

54  COPY  1 

53  MERGE  2 

52  0 0 

44  think  1 

45  LATCH  2 

43  ASSIGN  1 

41  release  1 

42  LATCH  2 

40  ASSIGN  1 


Input  Number  of 
Node  Output 

29,54  2 

54  1 

53  2 

44,52  1 

INPUT  1 

45  1 

43, pi  1 

41  2 

42  1 

40, cl  1 

38  2 


39  LATCH  2 


37 

33 

34 
32 

30 

31 


LATCH  2 
ASSIGN  1 

acquire  1 
LATCH  2 
ASSIGN  1 


acquire  1 
LATCH  2 


36,c2  1 

35, pi  1 

33  2 

34  1 

32, cl  1 

30  2 

31  1 

55, c2  1 


Output 
Node  Number 
31, OUTPUT 
55 

29,55 

54 

53 

53 

44 

cl  ,45 

43 

41 

c2,42 

38 

39 
36 

cl, 37 

35 

33 

c2,34 

32 

30 
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